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HMtCDT-Z • x F U~AH©^*H»ffcU *BS 
H»Hfctt-Y'<> Mfti/ctt^ h y-A 

X F y-A • £ ^ AX 



•5 > F • h7^) 



I 
i 



« v 



A 



\ 1 1 / 



J. 



(2) 

1 

F-)i7 * * hg^TS * 

^iB»W«:Hff^W6ti, ttl o*te«2o«±© F 
hl^iX h U-A|5l»!{t(D/c^(Dl0$/c«2 0 

F^xT • ^U-^OCX^Tt*-* • * h 
J8*fcU ^-^^-=e-FCCg^ft/clo$/c«2ot( 

**«war * s d: 5 kits xf * y<t n>£ c <t 
mzti. mmmmz, mmz> • ^>x*>* • ? T ^ 

<bic<fcorfPfiESti, F^^^li^xfA±-c*fjfflr* 
SAHi^>f^©7 7-f;b • ^^x^ F tux^M 
£ft, ? * • y^XA«7 7 -Ob • 

B»H«tt*a** • 7r^' *zr&*9 hCDi^fSm 

*± 40 
m>m3) m^mi^m^mc^x, mrt a i 
fc« 2 o«±cd f ^ ^^±<offlrejgjK f > • > ^ 

> x # > * xmmtt. tat v-vm&tt^tctboifu 



9#gfl¥ 1 0-28 3 1 9 9 
2 

o$ /cte 2 ott±CD F ^ Y ^±<D«ffB 1 o * fc« 2 oj^ 

< ifeio©*Bj3£»snfc^P^-fai^ yy^F 

& 1 rxDgity ■ >X£>;uc*tLT^ci ^ ^ • y 
KCtkLT:, If- F • ^-fw ■ 3>#-* 

> h^ft-eftcDiiteifc:^ • ^>^^>^rBuia^u ? 
^ft^ftco^a F • ^-"tj • 

ho 

^A - ^n^^ FCcfci^r, ^p-^^^^Xa^ 

(cL//cCi*«Fl»i-r43>e*-* * y'a^A - y* 
Fo 

^a • ^'n Kcte^r, ^a^^^^XAit 



(3) 

3 

[»#« 8 ] nism 5 ictetg© =»> e * - * - ^d ^ 

^A-^D^i'l-Kfe^t, y i7 • y*-XAii, 
8GK©3>#-*> h3WfilfflT*4*a^- F**T • 10 

[»«3S i o ] n*gi5 KEiW>=>>fcra-<i • xu 

©— SBi LTfli!©3>#-*> hfciitTl't^^ 20 
sci4Ult«3>t;a--i • ^o^a • 

* Ho 

y^A • yu#t> Kcfc-nr, fa s» d? • >*-XAtt 
#t> K 

^7 A • ^a^i' FKUbot, f B? ? • y*-XAB 

©3>#-*>fk:««u Wfflara«u^*««-rsj: 

5 teUfcC -Tin :>fc'*--$ • r^o^A • 

i 3 ] a^©^- * - * h y - A*ra«Hfrr 

- •y->y;U©#M7 :r -^ • * h D-AK:g-3t,>T 

Wmm 4 > # — '<;MWi*' <3 fTPJfiE 3 ti 5 x t- ^' i . 40 
1 o$/c«2-oja±©ffe©7 ; -$ • X h 'J-ASrVXi? 

• i-u-; £>#JtfUc»"5t>r89IU ffe©-r-£ • J* F D 
-A^M^-f • * F y-AOftHIilsiaWbStiSJ: 
5K?-£X7^:^££mSC£*^»£-f 

Clf5}?« 1 4 ] 11*31 1 3 (CfBiS©^fifetC*5^r, v 

CiSiRJS l 5 ] iiSjJBH l 3tcfBiS©:£i£fc:*$t,>-c. ^ 
X£ • «a ? f-- F • • x-i?*> 

F*»6©*fcKlEgr-0-cmS3*i*. SKRHtC-f 50 



#BfJ¥- l 0 - 2 8 3 l 9 9 
4 

[ mm. i 6 1 i 3 <,amo>?;mc a^x. m 
m-( summit* y= -it • v^-fuwvmzzA 
ax *n^7mkvc& ->xn htiz> c t *w&t? zfi 

iwm i 7 ] mum i 3 icfa«s©^ffiic*5^-c, m 

-2>r-f • X h D-A^r-'N- F")i7 • 7"a-fe ? tfCC^ 
3 >tc»-3 < «!B)$|fl|#JR(&«]E:3 > * > F tc&tt 

-b ? -^©a&a u - h terns? & £ -5 t,cm&? s x •> -f 

Xtb-yX, • 7s h 'J-A{JB#ra^>$-A;U«f« 

*SHa-5W 6ntC> ^ 1f> 6^fiS3 tir t,> -5 &© 

CW*^ l 9 ] l 8 KEIfi©*jSK:te«,>T. ^ 

fflB#ffl#M(* ^ a / < f ^ H^©— SB £ 0 T T f -fe X 3 tl 

[nmm2 0 ] M*^ l 8 «:fB48©^(c*si,»-r t «3 
a«flBI#M«. -9— F • • x-i>x> F^6© 

aj^f a » * • y*-XA©-gp-c*,aci^m<t 

[»*«2 l ] tmmi 8tciBig©^ii(c*j^-c. BS 

[»*3S2 2 ] »3J31 1 8 {CStt©*ffi{C*»l>r. B# 

X, &V>yMmMS<r-<T ■ 7 Y ';-Art©(4g* 

l-^*fc«2ojy±©ffe©y7 :: ^T • X h U-A£-7X 
F'J-A*i#I^f^7 • Xh 'J-A©Ma£|5JW{b3 

j; ^ «:r s 9 vt zmz. z c t *<&m± t *>-t> 

x^ • ? a i/#M«7"ay^^ m&O— UtLXT-7 



5 

c 2 5 ] m&m 2 3 &cf zmco^m^c > r , v 

F£*7 ■ U>^^CD^^ F^x7 • *5/U — i"a 

tea3>#-*> hrtTtonayf 5 ^ 7 * * f u-acd 

MlU-h^ Wil»B#JHk:«^r^-F'5r*r • 
y'r^>->r, j*?at • x F y-Att^n-f >*w«ji/ 

SB* Pifl# M » :/ a ' < f* y « i U * 7 * *fe x $ h 

4Ci4»»iT4m 20 
Cf»*3S2 8 3 fi*JS2 6 KIEtB<D#&«:*Jl>T> ©I 

a* * > ^flWKc ct o t» h c t t -r 

So 

t, *$i:o*fta^- f^*7 • *^u-**fireuT* 

nn^f-^ • x f »;-Art(DfiSK:s-^<fi[SB#ra 40 
mt. ««^- F*«?*r • *^u-*fcas-3<*MB$fH 

«a^- f^^7 • **s\s-*<Dm&&*m ^>*~ 
*> FccctorM^-rs^f-y ^<b v 

S& 1 n > F Gtfctt S(iSB#paffl«rfta^- F ■? 

C»3|8H 3 2] m 1 3 > * > F Kfctt &f g3@5HII 
ffi£fg2 n>#-*> FCCfcW4}gS^raffite»-30> 50 



«fM^1 0-2 8 3 1 9 9 
6 

r, jwccxfta^- f^x7 • *zs\s-2&mmux$z 

*> FcCcfcorM^T^^^^^i, 

r ags-r 4 x t* 7 :/ 4 c i t ir 

[fS3}<S3 3 ] IS5j<^3 2CcSB«©*ffiCc*j^r. J§ 

[If 3fcS3 5 ] 2 0CD^-^X F "J -A©HJB*ffl|«r 

SS*n>^-^> F 4*»B3>#-*> F*dtf«H»© 
»i ^7-^;u • F • f-^»>^^> Fr* 

e»nr#/c0i • ^ f h-acd^-* • if >^ 

*R*ffiO, «iffl3>^-*>F©4>tt<<k«>lo»v 
X^-^P^ - 3>#-*> F*Wjl^6tirc*4 

> F i*S*S3>^-*> F SrdtffflK© 
^2^7-^;i/ • ^e— F * f-»3>^-^> Fr*^> 
ot, H3fer3>^<-^> F«»2 f*-* • V-X^6iM 
6ntt/c^2f-^ - ^ F U-ACD^-* • 1f>^l/ 
M^>^-^> Y<fypti< 1 o«S 

mwLom 1 i^2CD*n*n<3D^-*ju • f • 

• ^e- F • 3>tf-*> Ftass^r^or, 

[fS^c3I3 6 ] 2o©^f^ 7 • X F !/-A<D|B*Wfl 



7 

mi j?4 r • v-*±, 

fill • K • *?4 7ffln>#-*>hr 

j*747 • X F y-ACD^r^ 7 - D->V')\'%mfrWL 

u>#y 

P** • p>*-*> F#l»ii^tt6ttT0>£fc<D<h, 
l£2^?^ T • V-Xi, 
^£n>#-*> F <fc*»S:3>#-*> h 
IS2*7-*;b • ^e— F • r«ia=i>!j<-*> h-c 

j?47 • xf *j-/*<dj*? : 4t • 

r • xf • x f 

<ot 

^ T • U>^y>^ • ^fA a 
[0 00 1 ] 

9£U>#y>tf • *P**£<DKI<DX F y-AEISHfc 
*5<fctfU- Fg^£l>ofc<fc5&C> -?)V9'jtf s 47 - 7 

*? y t -\ u a > re c * * s > yraacc a 

£p>fc: • y'utf^A * ^p^Att/HT £4>(D 
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8 

fctt. *»Mttffi2««3tifcV7 F^*7 • K9-f>< 

>mu^a^ • ^#~XA*«ie{t3ftfc:frffir 
sfltu mwKDtmztiz?-* • x h y-A^ismwt 

3*1, Mfr&^-F^T • ^p»>^ffl©U-F^S-& 
3*i*J: ; 5Cc-r€>^ffi*oJ:D : r3>f^-^ • 7*d^a 

• 7*p#* FteRprs. 
[0 002] 

10 fw^^Mt^^F^t'irtf • 7 

tt-rs^^^ r • u->>on!>* F ij-A«&fiuri»& 
©3&««Bir**. *fc, ^-ir-^hU-Att, mm* 
tctt & A ax £ >^1»«©£ 6^(cJ:otf- £ tm 

a^ jf>^nn«e©u- Frittf-r^o^stE-r^/c 

a&CC F^ v*>y#JH (tracking reference) <b LX i>{& 
20 ffl^tltl^c 

[0003] mmwit>tixi,>&Mffl< f c&m2tiz>®fffl 
4>z~^)umm*&m-zt><DtLx. 7 s - biased 

ata organization) , F, *o <£ O'^B/E (conve 

ntion)k£>£ 0 WAtf, 3 07 U-A/# (fps) 7 

[0004] ^f^7 • x h y-A*«yi-r*i#ccia 

30 C 4 1 ^<D^jH«, 2o*/c»*ti«±<D^ FV-A* 

HJdffc it cti but—me u> y y > ^3 n Set ^ &t-r 
^^•xFy-Ai, wicctttt-rsbr^* • • 

[000 5]*^WtMW PCi'D^OJ:^ 
ftfti^O*; F^^T • ^P ^ ^^cS^<fii ; ?: 

fco^p ^ • ^#^XA£^JJBL<rt,>.£ 0 f^-^-X 
F y-A • 7'Pt^«co^a y ^ . y ^^XACDfii 
40 tjHftur^ F y-A©Mi*^iu>y»;>^i5l«i 

{ tf * C i J&SnTffitC ftoWJ. [51 - F * T 

• ^P y ^*«*gtCLrtf*D*l4*^ 2ocDS^SXF 
y-A«fttfflt*S>f^>F^2Ji^S^ F y-A^c*fU 
Ti*fi:Cctfto*iSC4*s*4<D'C. »CCBI)H<bSti*4 

[0006] %\\<o^mit-^mxkz. x f y -a«$ 
nrc^p H i rvx^j • • x f »;-a±cd^u 
-tf>-r-->3> • ^^Ax^>y'fiiccffi^<B^fii*Sl 
fit-rs^p ^ • y*^XA**jfliorc^. vx^ • 
so xf y-A*s*±*fc»ff±stisi, i^Bifii«pc^ 



9 

yf=wT • x h y-A<D:/a-tz *ih*c<d*p * * • y 

±©«F)e©Bsracc, wa^n? ^tca-^< — 

[0 00 7 ] ^>ha»ra%a*n**ccg^ivr<,> 
^CD"C. mm&^X* • f-ZitcteJ^iT • X F y 10 

[0008] ^t^t^/c^-* • x f y-A*#yi 

[0 0 0 9 ] C Cr F^£(rate matching) <b«, 30 

-#;L> • • F^*rffiijru>#y 

Sfe^CC* v F^-^SSftfrfcSfs Stiii*. XFy 
-ACD^ffB^CDU- ■ ^ — -i ■* • ^- F 

9x7«X F y-A£U>#y >^§if(Dl/ 
-FiC— SLTt>ttl»»^#*4. CCDJc^&Ci^e 

[0 0 1 0 ] W^CCj^Ttt, a-77JU • • 

/^F^*T«riW«*ff5J:9k:-r*i, p-#jI/ 
WJ£ V - X L r c * Sfl«»«:i6-W -5 C <t oj 

^*7#ctt*wteKori>s<tttffi6&i>. ^- F9 

[001 1 ] mm&f<D r F U y h (drift) J ttC^U- 50 



#Bf!¥ 1 0-2 8 3 1 9 9 
10 

F©ti«», ^#ttB«*i^«rtTa^<to«>. /rai 
^n^-n^assw:, -**?>f • ^-^^ r^+^^-r 

&(skippinq)J fcitf/JWiiff* • ^-Jf* r»6 
<&£(ierldnq) J <fc t>5 JFJ-CJgft& 0 

[0012] • x f y-A*«a 

■r*i*«ciHC*S6«:Ji(©n«tt % **B*IBJ:7 u-A 

taoflMWB^u-AKaaft-rsmar**. coma 

<OI5l»!©racc<t<eCorir>S. x^~»££«/NR(c 

[00 13]7Jl/?^f^7S^m V7hS*T- 
F^><*ffiE88ttU ^TS5tS^mtciS<, -te* 

-FtastTSti, £<©»^ F^-r^asccafp-r 

F^^^r*M*tTtt5J:5K:r*4»»^'r*&. ^ 

<(D77''J^^3>^ *HJ*fflSCC4oCirwindows MT 
ffllS"C r^7-^;U . ^r- F (kernel mode) J tmttl 
CCD#IJ^^^^^rv^"7 ^- -V>Xj§l6jCT)^r- F 

y • vx^Afe«fiBWCcp|*<D*-F%feor^4 0 c 
Gcfc^ccffi^co^r-YT • x f «;-A*HJH{fciy, mm 

[0 0 14] 

feht^i^'t-F- F^-f^*««Cc«WT 
1 ^»lt7;l/fyr^7 • f"-^ • x h 

Eir-^f3>7'l/V 3 > (decompression) 

CODEC ^-^^T^U^ff^Cb 

[0015] CO<t5tt7 ^JU*BJL-if • F-CH 

- * • x f u-Att*©ttft±— jttt****), ^t©f 

-If • Fi • FiCDIHO*«U/W# 
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[0016] • F • 7 

(Dt 1 — £ • X h l J-A(Z)^iI^r|lI^t'r6C<brA€> 0 
[0018] *»W©J3'J©BWW, f-^^h»J-A 
F y-A • f*-* • *>^i/{c»Shfc«ftll]«I 

tfijis-ymmmw y^t^c^^xm^titcm^cy" 

~C&£ 0 20 

[0019] 3 6^ tBSg^$n/c 

[0 0 2 0 ] 3 *»W<oaj©BW», -Ji<D?U 

[0 02 1 ] *#WO*Offt©aW*J:tf*JMi»«T© 
3*i£4>CD<!:, *«9B«HJ6rSC:i{cj:0ftiff U#4 30 
*SCfE$S3*TrC>&-<>.* F "J ^ > r (instrument) 

[0 02 2 ] 

:/ny* F#s»tt3tiTCi4. ^n^-^^XAtt 

ffltts, • * f y-A©«H«w<tr*ffiE 
mmajmte*j-*)i> • f • f^ oris 

[0 02 3] • ^#^XAte3 0CD^ 

&StifflHB«rt>*. »<3D3>*-*>F#«fflr»*J: 



^¥1 0 - 2 8 3 1 9 9 
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[0 0 2 4] #JJ!B(D^P^ • ^#~XAr*iJJJjr# 
4Stt(DB#rafil«{4gB$^fii(positional time value) 

-c&o. ctu*^-* • * Fy-Accnit-r4«fm-/> 
wmmmmx$>z> 0 *acc. »3coi9Ritta4ini9n« 

(correlated time value) T * 0 > C ftfc£{igB#p f flfiI& 
£<Dl%fe&ffflB (designated time value) ?:PC^Pr 
*&£©ft»cMflj^«tt^- F-)x7 • ^P*^CCS 
-3< #JHl«IIBfflift«c!a«T*. 
[0025] ^o«^ • ^^XA4vx^? 
iUfcitB, (4IWII8M(Df-^'XFy-A^ 

(c. *-^5>^->f^>FjKnfe«s»««H*»|i«:ur 
8BS3*i*#, ur3S*«c#-f 5 

£(r*|*W*. CCDCite, rt*±3tl/cj WFlffl«r v 

• *P*£ • ^^XAi3>^> H*fiBBB$ffflffi 

[0 0 2 6 ] <f-% • X r y-A • U>^^^/c«ffe(D 
:/P^*lNcH«T&a^<t&£^- F^x7 • ^p 9 

^<D^a^p H iffl«, Mi^n^f-^ • * h y-AcDu 
jg*>«>T. iaaB$ra<BicA^rv^u>^y >^ • ^'n 

42^^<D*^<D^SU-F<?:J:btS*r6C<fc^-c^^ e c 

necefc o . h cd r f y ^ h j 4fflii^oiEfiltc 

*j»?r^4cDr, u>^y>^3n/c7 r -^tc@So/c 
us*^^. £ c <b < Hmomte&fr *> c t tuv # 4 c 

[0 0 2 7 ] *f*{C, tBM^rafii«fl6<7>^ a 9 ^ • y ij 

^XAiMa^^y F^ntff««»«sqi^6 

<fc^tC, ffi©^P^'^^XAj/t»3>^> 

h **Mfflr ^ 4 J: 6 tc ft ~> X I > tfhtf ft 6 * t > 0 ffiM 
B$p a 1«T V % • *^<U -is b > £ LxmZti&ttc 

\xT2*:xztiz><Dx. ^?&mm*m>mx%>*). 
[0028] mmmxzcxwRLx^&m&mm 

rtt2gaW^- h 3tl, i^6(7)^4>, PC^D 
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JSEBSffl (designated time) tUX&mL. ffe&fcM3Jffi 

[0029] loscDjgiBtt, «m^k^n/c^ 

ttr*-**- *-F • K^-f^*a«-r*-«iur io 

> h <D r fc*> • 7 7 ^ h D (pin factory) J *tr?F— 
±<Dffe<Db'> - ^>X*> h£ffi5««SttS««hT> 
ft, a— !f • F ■ x-^*> FtC!H€>ft <Tfc v M 

[0 0 3 0 ] n>^^-YT> r • F^ ^* (compliant d 
river)£^T&C££#ST&1f- F • • X 20 

F« F^^^^CCtg^^ababilitv)^^^^^^ 

l^/cCt*Sft^ COCCUS, M^ft^-^GD^ 
fc&2*:-F, 8BRbT> • ^>X£>X<DA#*fcteffi^ 

jj&JbZ. 30 

[0031] jL—y • F-rntf 

«±CD^>^^^T> h • F^'^«1l&COt,>Tffi£ 

sxf-^u r^r©F^-r^«ffi©js^*»^*r, 

[0 0 3 2 ] • -OsXZZsXZttC. If- F 40 

• • X — >>x > htt^y77 • Td^r-£(buf 

fer allocator)^SOf> • >X * > X "CASSM" £ 

iKs^fts^^^^fe^JK-rs. ccD»^«>. emit 

[0 03 3 ] *(D*t, If- F • ^'-fw • x-*P*> 

H*. £ h*7^Ai(Dgif> • 7 7 ^ f ytcs^r 
>*m&<D— m±uxr-$ vtmmy * so 
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>x*>xtt ry-r-Obj ^<o^> FJi/*R*fl£j«AtH 

ttF9-f'< • -f • ✓ FA/& % Jgi^b> • -f 

[0034] s etc. ei»{cff«stiyfc«*ie> • a > 

a*e>*fc»i rp ry-*j - tr» smsr 
*R*©*rjB3£sn, as»tr> • -f >x*>xih<d« 

tfij^vvr - ro^-**fe-3A*tf> • -r>x^>x 

*©v-x • t*> • liccauaiAsfffe 

[0 03 5] 3>^-fr>F • F^-f^sftys-rsfc 

aMU*, F^-f'TO»Stt* JL-lf • FOx-^x 
^>o windows nr*^u— • e>xf*A±r^il3 

[0 0 3 6] £!^f**«F5ET S G U I D (qlo 

bally unique identifier) ( 9 P— y<;l/CC^x— 
Wf) t % m&\ft<D&m&XlsS>b<DRU I D(relat 
ively unique identifier) (tBSctWCCa.^ — ^ftffllBfl 

I OCT L s (10 Cbntrols)CAtH^ ::l > bU— )l) 

/cwiwa^en, i octl«, «^<Dtt«ttt* 

[0 03 7 ] #H«©J$BT?ra. 3 ^ -f ^©«^3W«ffl 

snti^o o*o, ^o^^-fSi^ ^y^F$^ 

^ffl*K^0«fc 5 ltl^(D^ST77^<h- ^CC# 
— CD I OCTLI:ifflt§o ^V»Fi*B t 
SU^r, y< v V y<DVvv ^>>^(flushinq)^^CDJ: 5 
«C % F^-f^lltf"C#S*^U-^3>*^|-r4/fc 
feCC^ffl^n, f-©IOCTL4flEffl0tfi^©^V 
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ftK -r— ^ • X^- ^-^3>(data starvation) il^Q 

1 -3«:4$S<E>-f 'O F£^*-:/;Kenable)&tT£/ctf> 

(disable) CC"f &fc#><Dfc<Dr#> 5. 
[0 03 8] *^*fl6ffl-r Stett, AHi*©JfflI^^U- 
^aX^OIOCTLi, *^(DGUID, RU I 

^^rftfflurggjf&^n^o F • *- F • 

F^-f^rtfy a — A • ^n^-f«rH5E-r 

i&^o^-fOiocTLtKfflu *y»-Aitt 

(Dl^O^fRU I D**#y*-A • :7P^-f*S* 

[0 0 3 9 ] V-tf- b 3h4*^CCoc»TM*-i-*CC 
tt, ^JI/GU I D3&s»®B**^^©«jEI OCTL 

OCTL, Sfc»^>h'-f*-^HOCTL)T 

I DCDUX h$$£3ft£. &£5l^ftCD1t#~ F Sft/t 

SCC». i^GUID, 1^^7'IOCTL, to<fcV 
m&V^Vifi* IN*- h^n/cRU I D0»J * h^rilT 

[0040] iafflJU^^-XASffifla-rsi. 

*S*r#S*:ircSc<> F^ !^«W«K: 

F\ fcJ:OV^>h^If^Ci^^ 0 

@<0*^*5E«U *Ot^GU I D*«0i3T*£<!: 

-hSft&titg <o*0 % 1N<- r$ft&GU I DiR 
U I D©l«**tf9Ci) tW»r**J:5«:-r*t. 
tf- F • ^-^*fW»x-^*> F&£©, P?fflUM(c 

[oo4i] ttamMflL tta^nffi. 4BH&KB$m 



(9) 1 0-2 8 3 1 9 9 

16 

• *^^*5%-rGU I D* h y>yffi£|gSLT* 

y-A&c2-3i>Ti>s 0 !g)ffl«Flffl«, ^-F^7' 9 
PC^D^*JtSUWOt, PC* 

o » *^#fis<h LrttMurc^ojmajwr**. 

[0042] 9XXv9 • ^^XA^; T^X^J ilt 
[0 04 3] *»MO±IB*$ J:CK*©flS©BWi««W 

arosttw *j J: cf»5R©«Hcc» l< Btt s tir * a 
[0044] *mio±s&to&&*<Dib<Dm&± aww 

[0 04 5] 

if • ^e— K (user mode) J <bl^ffllS«, a— tf#*l,> 

40 -if • b<DW)l / El''<MX'te** Vta • U-<;U^S 
H2^3WJJ(©T^y^-^a> • ^a^Aj/di^O-fe 

ffi^snso^f^RWr**. ^x^a- yv 

[0 046] fflifrMt^ntc^ r#~-*;u. 

- F (kernel mode) J <hl> ^fflfgte^L-lf • ^t— FCD86 

50 ff*-FJ:0fcl»BaWWItK:d>tt^ ^-<u-t-^>^ 
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• FCDy'U^ A^/c^P-feX^i Utlt ^ 
-F^xT • 3>#-*> h*W«PT4y7 F^xT • 

F 5 -/^(software driver) {-^#j£0T, # 

— • F0^O^7 A^7*-7>^OT3 

JRSflttr*s*», &5t»tt3L— if • FO^O^A 

Ftt. ^fliJ(convention)CC*f-r ^>7'n y~?-?<DM»!EL 

Uttt»J:^«:-rS) o • F*a*>T8iJ©JB 

f§£LT, r h ^^t* ^ F (trusted) j 3— F#S*S. 
[0 04 7] $WfflftfflU6ntl^ r F^-f ><(dri 

ver)j L^smmte*-*^ • ^-FmtTsns©^ 

* £ 4 y*n ^ 
•T^^t^^o F^-Ml*, SKflMB^ $>Z>fcm<D 

A-F^xTisa^etit^w wut*5t 
[0 04 8] $Wfrffli^nri^ ry ^;i,£(fii 

ter)J il^fflSttV^F'JxT' F^ Y^rtfca^ft 

tt&ssn* few, gii#^>F^^^4iu 
7F^r- F7Y^iii<^©s6S7 

"Ctt, 7 ;b^il^fflfg«, ffiSg (decompression) & 

5 **i3W*-*Ji/ • ^e-FTStffStiSv^ 
- FrlSfT ZtiZM<D7u Free SrtHC 40 

[0049] *9!in»rffli»einri»4 r p^-f ^ • * 

Zfi/*2 F (driver object) J <tl^ffllS», V7F 1 ) 

xT • F^-/>**«fflU cn^^xfA • yy-xi 
[0050] «ittffli^nti^ rfn'^ • * 

y^x ^ h (device object) J <h l^ffll§«i^X f-AtC 50 
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J:otSUn/c>XfA' u<;KDi>f -ff -f©C 

<br*o, cn*»j»3^i«t?*4F^^^oaitt©— 
t&kisx?-/* • yy-^iur*D6-tt. K7-f^oat8 
ti©->^fA • 3>#-*> F3W#jffl-car*fr£5* 

xtcdp- FWiiaawwKfwisti*. 
[0 05 1] $?3}fflirffli^nti^ ry T ^;u - * 

F (file object) J <fct^fflB« v f^VX • 

^isnti^. 7 7 ^*^x^FttF7^. 

ho^t^KM** 3 
&„ 3e>ec v y • 4rzfss*9 bit. J^ituoyw 
;b • hJMKLOy r -Ob • t^x^ F©ff 

«cB*^c r^j i}8j£3ntt»ti«, S'J<oy r<oi • *y 

-Ob • *yt>** F« t ttStffKCT*-* * x F y-A± 

[ o o 5 2 ] *?Biu»rJBc»&ftTt>4 r^-^cdat 

a) J t^Smmt* iI8«3tlfc*-*;^*-KCD 

y ^ ;i/^«riiurteas*ia— wot»*oc £r&& 0 

MI D I ^i^a-r^f'^T • ?-%%&tsf)^ m<DT 

~c\**2>m&&$>z>a mz-tt. • ^r- f • y ^ 
^? • ^y^yn-fe^sofap^u-i ^3>rtffl$ti5 

y ^^©^fA^6, i^oi^^rfij^fw 

[0 0 5 3 ] ^WmW^te. *mW<DmWte, Microsof 
tceSffi«) tt^6ffi^3tin^Windows Mr 

CDJfJ^S^T^/c^CC^, Windows KT<DAtitfjT — 3r 

J:c>* NT Jr^v-r- a >^ • ^x^A^ccoi^r^ 
Ltc0MWt Helen Custer^ Twindows hfKDF*9 

SP (Inside Windows MT) J (Microsoft Pressor ) 

[0 05 4] F^^^*j<kcfy • ^-y^x^ F, 

^^•^i^K F^-f^ • *y^x^ F^i' 
<DisX9~A • x>tm ^ CDtiTC7)l^ B >3"e^, Windows 



(11) 

>F£&offecD^u-fw>^ • ^xfA±ri8t 

[0 0 5 5 ] s-r, Hl^JWLrittWrSi. H*<D 
^XfA«. 1f^> K • f-*?<D7s b y-Atf** * 

F?<f:/*>6K»B(!>. *©*9>F • 
>yy> ^(rendering) t£5fc(D^f^UCt£oTX 

-fi?*Mb3nfcif9>F -if>^KD»JBr-y-»>>F 

^*6«J5SSti, cn6(D1^>^l/«r-^ y*-v 

K7>f^'2 2«f F^-f^-^-F^T 
2 0 chtBSffJBU a-lf • FCDU-^(reader)^ 
P^^A - 3>#-*> b 2 4<DWJ8PTCca^tlTt* 

h (controlling agent) 2 6Kt^ 30 

>F ■ f-^©i/>^y >y*tf ^yt«>ccs&S3>3j< 

-*>H*«i-r43W, »W^7f^ja»*BCdynairic q 
raph building capability) £Tl>&W&&, 

F^*r * n>#-*> F^ttffjccSK^re)*! 

V > y * /ttef6<DMSMK£S«r ^ £ <fc 5 *CT£ 0 
[0 0 5 6 ] y-y • :n>#-*> r 2 4«t^U-r 

>^ • isz?-&<DmmAajimm<{y $7 

> F • -5*- 7*2 - bU72 O^^Sc^BXo 40 

:x-tf • F • ^n-feXCOT F (address 
space) (D— SBiL/r^— !f • t-Kt«3tfc^7 7 
7^CAni)#^^1-^ 0 ^CC, f r ^>^U->i^(decompr 
essor) • n F 2 8 Wffiffl^-^^aCCjaL 

(decompressed)? * » r tCf*ffi*T £„ H^O 

•fe X KlfE (processbehavi or)S^ ^ *7 x X A l> T 

a— !f • Fr*fTS*l*. 

[0 0 5 7 ] $J*y j (effects filter) 3 0 ttf*— 
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fctt, «3>#-^> h 3 0(J^XfAAfflMH^ 

5£-r£ e c<Dm^i>. *-*ib-*-F/jL— if 

F«Hi*SS«Ur, C<D«»#fT*>*i4. 

[0 0 5 8 ] #j*y ^^32 KH9UK^P-b 3 4 * 

> F 3 0*>6£ir-££:nt:~U *cr>3f-£afcgi:7 

yy F-)x7«3>^^> FW:^- F^r^'D 

F • y'n-fe ^it±r^f $n^'>^fA • yy 

[0 0 5 9 ] > F 3 0 (D^SCD^TBtCC 

cn» v ^ccc, if^> f • u>^y >y • n>d<-^> 
^y>y- f^-/^3 8KHk£u c*i*«wr*^> 

F-U>^y>y- F^-f^tt % MS^n, 7^;l/^ry 

u>^y >y-r4<t5ccif^> f • io— F4 o*»jiarr 
^> 0 «±^6SSCCffl»ShiJ:5K:. a— if • F 

tf)-*)\, • Fw©Raitt^«H#aE-r*te»{c, 
•y*^> F • f-^©u>^'j >y*^B8»{burc»s. 

Ftsfcfcfclff^ ■ X F y-A<7)J:^Cc, 
[0 0 6 0] ffifflfrffli^nri^*^©-^ 

Windows 

ShTl^. ^ltt, ^-1f • *-K<DAP I-C^O, 

iLT, Fy^^e3»^S^{tT-^7*^^ + CCSe^ 
b~7j^£ftf&?&<Dttm?ZJ\s-*>£7-Zffim 

affi^wffl-rsi, was f^-/^h«*i*««©t- 
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* r ? * * tcmm u x mmmmm? & a > ^ -/ y 

F • x-yx>h(JNTx^f^ ^teitfAttW^ 

- K ruff *cosm^ ^> X ^ A £il L T 3 > 7* 5 -f T 
>h • F^ chit ffeO-T^TOAtH 

[0 0 6 1 ] *jaBt«fflf SB lOV^f A<D7-+ 10 
>h44tefcie>3*vct,>£ F^^CCjg&U 7=-* • 
c\ i/>yy>^s^(c*'*^ • F-efr5<fc5 

[0 06 2 ] co«flcr« v 1t r >> F • ^-^^mi^U 
/Cct^CC, f^X^- F^-f'*4 S&Cfc-oTfw • 20 

tt^wX*- F5>(>*4 8*ft(lflPU t£3fcCDffi^#£p) 
D<fc50CNT»(NT layered)Aa#y-*^*^+*C|S 

T B AUMjy - * ? + <D-U t L X m&ft>tlX C > 

£/c#Hcm^6*rc(,>£ 0 30 

[0 06 3 ] FvJ'*5 0teLTFXmJi?Z>m 

r*¥*r^jj trctmmmzti, «ifflix-^*>h4 4(c 

««7^il/^5 4K:5l»ar. »*7 -f Jl/drtM&S(Cj£ 

F5>f^5 8{C5I*«U CCD F^-f^tt*'» F • *7 
-F*««IU f - ^ t:-/; 6 2 *s6©t-) > K i 40 

- F <t ft - *)l • * - F 4 < & 

«ffli*a-if ■ *-FrfT5£»»ec4*-^ 

[0 06 4] ^CC, H3*#flSOT8fili-r*. B3tt, 
^S1"P^f ^0"C^^>o F^-f'* • ^^v 1 *^ h 6 4 
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T • =J- F ■ ^-^**Tfc*ecft^R3ft*. F9-f 

0, K 7 a' • t x ^ h 6 4 X r A tcg^ti t 
thtf-F3ft*F5^<©^fc£<D^- 

[0 06 5] $ij$jx-^*> Kc<fc^ri£fi:K:y 

6. -6 6, ^Affl^lf^ h «;««CfR«3ti, C 
^x^F64<bfAVX-^^x^h6 6, -66 

n raH##**x6 8-c*vr<fc5K:, F9-P*-=i-F 

[0 06 6]S*K«Ctt, f^WX • htttt 

h X&Ztitc&J > x Z > x&t"^ J x • 

[0 06 7] <frt4% . 5h>r^^^ HS©Wr«, 
• ^^^^ h6 6 B ±tc« v ^t^;w • *^ 

[0 06 8 ] 7 4>\,2'£:&m-r&MZ> W»i-^*> 
F*fc««OJL-1f • F • *5-f7> FBAtHAf 

^ h y «firtrMfflpi(i8«cf /<>f x±r 7 7 >f 

>F;w^-if • f • ^7-rr> htcig3n€>o 7 
r -Ob • ^^^^ mSBSK r^j ^ T ^;u * + 

7is*2Y*mfe?&^tiiC&^xmmmtm%^v& 
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(sibling) B8^fcf#oT<,>5 0 

[0069] Vy-ito* hrt<7)3>f- + X h 

it>ltc^i>f ^ fwtf) rt^Si(state) J tt£*>6 
ftotl^. a><5r*X r-tiNRfctt, ^^fA«fi 

>(validation)£ I R P;U-^> ^(routing) - 10 

[0070] «titr> • -Y>^^>x*aet-r«s/cd?>tc 

RS<Z):7y 'OU • ^ bteWmovr <{)l> • 

7 7 ^ • tfy^*^ h7 0ti?n/c7 -f Jl/*<DJ8iK 
b*> • -f >*£>;**S0, 77^b^^x^h7 

*:/5/** r 7 2 £ 7 4-Cl3tl/cgif> • ^>X£ 

h7or«3*iri*s) tcAo/c*ir\ *c*>e>uis 

[0 07 1 ] b*> • >^dr>x*7 -f • 

[0 07 2]^?77«ary^XA, £>fS>^- 
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6tife^?77»St^*iXA) IflORJINffitt. & 

0, Ch6©^> K^tiNtCreateFile^^^^X^AA 
P I x-JlZMLX*? YT> rKiM3n& 0 
[0 07 3] 77^^* r^<D>^> KM2# 

• ^e- F • F^^<hjl{rrsfctf>«: % $ij»x- 
htt£©;x-if • F • ^-Y7> RcioT 
&ffi£n£ 0 7 7 >f;i/^^x^K f^vx^y 
t>**k teicXF^-/^' • KOKJtW^* 

• *W*>? Ytfrtjx • F^ 
-Ta< • t^x^hStMl HRKD K^-f>< • 3- F 
CCA^^>h l J • tf< > r (entry point) CCiiMT^C 

<fc#rs*<fc5ccrs. c<D<fc5&x>r y • #^>h 

V t<DT$<D*-7i>*9 riiBS (pathway) ±CC^>^>^ 
AtU*t^7-A^ I RP£fE 

&^-^>#:teJ:ov\*y f"-^H > • y^^X^ccfi^o 
r I RP^riEU<^-^>^T^<!:^CC^ffl$nS7 :r - 

[0 07 4] V y - V tifoCDi/Xr-A • 

^>C<h-C$>^ 0 C©C<bW. ttSlOT^y tr-*sa>& 

ccm«*ea&r # ^> c t *stt*r e a 

[0 07 5] F7^M*tBI««t4t*lSi 

«cS««xuy>h©ioiur, f^-fx-t^x 

• ^ ^CD3I3 (depth) ^^^y-^7)^^ 0 cn 
««Fffi©F^^^>* • •XrfVx.V h<Dl RPXZy? • n 

<DI R P ^rftffl LSWifiU"Ct IRP£SIJ*K: 
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[0 07 6] ^CC, H4^H6«r#«SLr»WrS. 0 

(i/ORequest Packet: I RP) <7);l— ^> d/* ojf^t 
tS^fA- F9-f>< • tZfZsxtr K fv^X * 10 

»3>^Ltl^ 0 B4«, log/t«:2oJ«JbCD 

tt:a-F**rF^><- *-fv*.t> r7 6£^uri> 

£„ F^'* • Windows NTT- * 

f^ttty7F>)x7' F^>*B»##JJi«Ofcfls 
^>F^^<D#«a*S^L/Tti5o C©^Jte©JBJBCc 

plexinq dispatch function)7 8&£ffr&>^> F^i It 
K7^a'. t^x^F7 6^6#13n, 20 

[0 07 7] HD«fc5 0C. F^ -f F*>6 
cDffecD/^> F5t*v;l^:/u*5/>y • X^^fS 

(mz.it. saxo, ^-r^waiair) 

7-*X hflMRfctELl^y f s ~2'a> • 7"-^U*^L 40 

[0 07 8115^ F^^«#6#fc<fcoTjJrSK:<fc 

>fx ■ x^x-r>^3 > •x'J782^t^F7>/A'. 

h8 OOfA^^ •i^Xf>'>3>'X'J78 2f^CD 
^i^n/cD^r-^s >CC«. 77-/;l-^>f7'-A'ij 
7^— >3> • f-~:/;l/8 4<hn¥«ft. Vy 4 

h * ^^7*8 6 cdx h y^ysas^n**?- 
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Ha-r&f^^»F^8 8^(D#«8#«frnrc^. ft 

^-/7 , .;NMjf-^ 3 >.^^84?:ffiffllt 4 fp 

■Br*. 1**3*14* h «;>y« i r pmas^Kjaa 

3ft, a— tf • FCDNtCreateFileB8^3-;U<f:it^: 
ffifflSn57 7-/^*Xhy>^6i!l6tl4. NtCrea 
teFild8an-Jl/«, b*> • >f >X£>X$fctefi&CD^ 
#~XA£m&^£/c^C^-1f • FHIft-feJKD* 

[0 07 9] H6». r^*T- F 5 

•i'J792^feo77^l- h90^l 
tl^ fl #Itt7 7 • n>-r^X hi»J792^ 
6 I RPS*^> K7 • f -^1/9 4MfbtiS. IR 
P^9 6<DmteZ>$J7'teft7£<D^l' F798 tmM 

fig«CCDlf$8*ffifflUriEUt^^>F^{CT^H2X-r 

El/WB&'O F^*9^-rS»^K:W, 7 y 4 >V 
-24? • ^yf-v 3 > • 7— 1 0 0 

0 4*iAoW4. ^7 7 • h (o* 

3?>f7 , tt77^'t^x^h'^-/7'10 

[oo8o]»c t B7*#jiaoTifti!rrs<fc, a«fp 

(D-f>X F U-^s > • X 
f^*106t, -OX h-;U • yo^A«jfi«J&v 

f^^> F-7»«afflv;U^^U^t/>y • f^A"^ 
iMltl^o SIttec. F7-/A'.t^x^F7 
BCD^CDffecD-r^r^^ F^#fi8«, W5E^>F^4 
^W^mfe* & OffeCDjfl.ffl (generic) v;U ?7'lst>i/y 
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*jh*®#J£r^> F5^;!/-*>^£C<b#-C# 

[008 1 ]^ ^r^'108t, v?F-5*T- 
F^-MSWT^JIfes- F©-f>x F-JUD— gPiLTft 
JSSft/c&rv^x • *^x^ F«> H5tc^f cfc^ 

-c\ i rps*^s«, awa^-fx • & 10 
[0082] 7r^i/ • hmtmzti&t, 

MWj: I RPf^A?? • ^-^l/9 4WfWJE3ti 1 
h • £ Y 7* • ^'Jf-V3> • -r-:/;H 0 Oi- #gfc: 

77-f;I/« ^>«r*X Ftt«9 2 4— J*CC4$jEo^r-S/ 
[0 0 8 3] H8*»JBOTBMirrSi, Httff 

3>4-€-CD^^»j7 :r — ^a> • ^^^XA^t7 P — 

3*rCl>S. 1 2T, jl— !f • ^e-F ♦ 

tX«7 7-/^-t^^F «r^J5S-T4Am*B** 
3£ffT6 a CCDAW^f^S^«NtCreateFne(D->X7" 

aap I *Pf^ffi-rc4«:j:or?ft>ti4. xf^'i 

7 8 KiJHiTS (B4#I) . 
[0 0 8 4] v;U^:/U*^>y • r w X^' * *«fiB7 40 

f*X F**fT*>*l. a77^^'*^x^h^*^f 

5 aspJK s ft c stf ^ yt^tcsMSttwwtt 

I RPgf*<D4HC&£;^ Clitic— If • F • V'O 

F • 7'D-fe^(J r$Bj 7r -O^ • :*:/>^^ 
S^>F^*f¥fl»*a>— SB4LTJBSU NT^>Xf 
A« rjgj ? T Jji, • i*:/^*^ F^<DIEUC>#JB*t> 

[0 0 8 5 ] f8^7 • *:7>>*# F#ftW*i«, « 50 
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•;>*fgfcg7 8M«0J5:f^'^ • F8 0*>>€> 

©f^X-x^Xf»3>8 2MLt 77^ 
;l/ • • n''Jf-^3> • -r-^8 4^Xf , 7 7' 

1 1 8r#M-TS (H5$jffi) o ^y^-S/a> • 

^«fiB7 8Wfi*©t|J<0X F >J>^7 7^1^ • 

h . ^^7*86©X F U >^4J:b«-r4C4CCj:o 

r % xr^ 7'i20t77^i/« h • 

[0086] xf->7'i22rxh Scuri^ 
4r7^tx$n§ 0 -HSlLr^ttwtitt, fB3EB#tt 

X^v:/1 2 6-CJE53ft4. ^124t7^-fe 
X3ftfcfl«fc"> F^^f^'l 2 6t77-/;b • * 

n/c77-fJi/-^x^hMu t 

>F^«, KWccffjflELri»fti i RPfwx^?- • y- 

;b - n>-r*X F 9 2©tf , CCffja-rS. 
[0 08 7] 1 6CC^^r, myy-oi' 

r^;U- F^fit^i^f ^7*1 1 6tHI 

W^n, -eti^fiRB^KBBilTS I RPCcAoTt^n 
« v r r ^X^^^«tg7 8«jH^ 

r-f;l/- ^^i^j:^ F 90^^077^^ • n>f-^x 
F9 24ffifflUr. ^f^7*l 3 0-C I RPf^^'^ 
^ • f-^9 24#MtS (16). m#B#<Dfgl^ 

^7* 1 3 2 "C7 7 'f ^ * % 4 ~J* * ^Vf-isBy • -r- 

• 7WX;<^llffg7 8«. ilBil^DJ:^^:, B*(D 
tp(DX h U>^^^7 • *7l>x->? h • 0 
2(DX F , 7>^4J:l:^*r^>C<hCc<J:orX7 : '^^l 3 3 

[0 08 8 ] X F 'j^^-SUri^iXr^'l 3 
4r«8fr3n&<^ aWttfWS^VF^X^^^l 3 

8rr^*fex$n^ 0 — mummm^ iftgsmz 

Xf'>7'13 6tj6§?ti^ a«I«cfft6^>F7«:tt 

^fi£^> F^tt»flXm£3ftfc7 7 • ^^e^x^ 
FOffU^ I RPf^^^ • r-^9 4^fW 

»fU<^J5RS*i/c I RPf^XA-^ • ^;U9 
4 #M££rL < fftSSnfc^ r • ^ 
F90(D^T-f^- ^>r- + X F 'X'J79 2<D*0CX 
r^7'l42tMt^o fttf^*Ci«, »t>( 

m&b. H6cc^«rraD7r-fii/-*^*^F«jfi*s 
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[0 08 9 ] «jftfcT> • ^>**>***f1^3hS<!:l,> 
*#>*<DfB3ES: Fj TS7 7^W©t> • 

Wfttftt S6fC, C<Dm<DJ Jj-XMZ.. 

<t&f#g-C*6o 20 

too9i] 3>7'^^r>h • h*7^^ii, « 

TS1R34VC<,>* 1 ocD^^^XA«^«r >f :rA(D 

id (ffl^wcc^-^aajm ai^eftw 
ai^«»*jj:c>*jiiR$n/cRu i dt^a 30 

I OCTL«*8ft4fflr*£. te©Sg»-C«, 

«*coflec^**saasi»^*J**. *b&d i octl 

tt, «!R3ftfcxuy>F (RUIDittffltSJta 

^>F^«caBart-4. 36cc, mwy^tr&Gu 1 d*> 

<fctfRU I D«fc—liK:«0T. *0«tt*«rfco4*L/< 

fleers c 

[0 0 9 2 ] MZKMte* -/ y^^OAr * JUi^r* 

;l/ (^fi) loOIOCTL^D^f^ 
nv>K4"set" 7*DAf • nv> F<h*EJSijLTlr> 50 



GU I DiRU I D©ffl^^6*S*^SIJT-(set i 
dentifier)CCj:or4$5£3tlS. 

siiwv? Fwai^aso-^o^-^aGu i d<l 

[0094] js<> bmsit* f'Ux&mmtos 7 s - 
tf * F • r^y^— >3>rffiffl-r«tffl3pjr*4 

flt^/c^Cfffl^n^o 2oOIOCTL*s«ffl3 
*i> lo^^cD^^>h^-r^-^;utc-r^/cd?)(Dfe 

ctrsfc&Qkcrc&s, ru i Draasosftfc, ^ 
fljr&c<b#sr££ 0 

[0 09 5 ] *^*«ffl-r&fc«, Affl*»^U- 
^3>«#®D I OCTLWOl^G I D, RUID 

£ 0 WfLtt. W> F • #- F • F^<Y^r;K'; rt-A 

o^PA'f ^l^IWcCGU I D, #lf^-A-^P 
3?r L l> # "J * - AStll* GSfflr £ C £ feiftRttCCtftt 

[0 0 9 6 ] if h sn*»^ccoi^r ^ ^ ^'Jccis 

*7ik-?fflm7 7#£&i>'-o. %$/E<DMc3& 4 y m <o I o c 
TL y'a^'r^ I OCTL, F I OC 

^yW-/, ^y-> F*/c«^-^> hccoi>Tflg^*r 

^>CC«, I^GU I D, m&ZJy'l OCTL, X)l>R 
UID. fcj: 

h ^tl^RU I DCDUX F«Affl^^U">3 



(17) 



31 

><D^t LXMZtiZ. C©';Xh#>6. 
-f- -f • x-£>*> Mi. H$S 3 ftTt>£ 

[0 0 9 7 ] GU I DrA--d»fcW'JSftfclS£<Dtt 
F^^fS^ilJ— F • /<— f"-f tWP*-J> 



10 



ttofcatte£4>J:9&ir---F -^-tw • s.-V^y 

(*i ) 
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[0 09 8] c c-ciftB^ur^-ss^fig^ii^-rs/c 

tfXCtt. 3>^7^7>h • F 7 -/^'tt* &«©*££ 

-2. <b * cc«wrcs < ^©fi^ts$B*^i/r^ 
^brf^3n^HK©gi^f> • ^>xir>x(cM 

[0 09 9] 
[ftl] 









SRfcfv - 


x*>x£vx h ot»a*. £BJJi]R]t6fc:*-f rat: y • 
affiffitr*' • -f vx* vx©*g»T*{± 


mm. 

4 y X5> yx 


tSttv C©:7a A-?-.* Its.- 1 


T — $ • 

"7a — 


• 70— ©?V|3)£«J X 1- UTV>4 (flRfcT, 

7* ju^*>ethA0T-5^iS]) . 



[0100] 



[*2] 



33 



(18) 

(?|l<Do-3#) 



1 0- 28 3 1 9 9 

34 



<\<J> r^V » ^(bridge) J ©i^K, Miloyetlfc 

I RP**aai1-*i:*<D»«try • -i^xtyxo 
iaa»cMT^. ^y^iiangr* i r p&gfs-r-Z) 

<9tC*fL. y — X(± I R Pfc&OjgfZlftSaS 
7 7 Y*o/Ut7rY;i, : gmw57-nAf^ 



[0101] 



30 [&3] 



35 



(19) 



ftm^ 10-283199 
36 





7--^ . 7*~v*, h&ftt£(ommv> - >x 


■i 


fiSfe n? (Pi t&L fc-i-- 1^ V . "7 ^ ^ L II Lmj^jl i , , , 

br^*. -r >?y*.-xmmv> • 77 n <;& 

38 b T jlfa T- # Z> f H 7 £ (i x — ^ CO 3 -C 7"C* $> •& „ 
WAIf. MIDIf-?, CDU-i?^, MPEG 




if <o <fc ^ tc . t»a*»<Ei&«iisfcs©^ffi*fcfct 



[«4] 



37 



C20) 

(311 ©o-SS) 



#Pfi¥ 1 0-2 8 3 1 99 
38 



S * * i ^ t^ 1 " * fc 46 ,;: ® m $ n * • " • 

7 y^S(aata intersection) "/DAf^tif — J' 

J* * ^— 7--f HiUffllJ-— M±> x — > . fjjr 

&JS 7 * - V v Y ti » 4«h'7-f A(C i,TS«h«, 

(virtual) 

*^ »^77«:Sit5Lk4)Tt4, 
C07D Af ^i^®»7^nSS fSSO 



[0103] 



[*5] 



39 



(21) 
(»2> 



10-283199 



40 









MM 


U f- y p (latency) tfgtjzT' J64^-f V h 

s*y y-xffiffltt?a©-e-Ki:, nfT«a»csjjtr* 

ryy a-;K P rerolled) J X'&ZAK Ctttt^P&tCtt 



[0104] 



[*6] 



(22) ftm¥- 1 0-2 8 3 1 9 9 

41 42 



m^tm 


7-r ;u^ic±orga$n-6>; y— 7,^<ot->*>x* 

leiSbT^D. 'J y-^tUST©l3f9 (arbitration) 
•fclATy -< Jl/*(;:J;-o-r;{£ffl<*ft<&. CftlT'oAf^! 

j.- *? t. v h a pis 'j y-x=5r- (Dftfem <t 

1*^T<7)«iBtf > • y.x* yxtc^^M^^stry • 

C <D® 5tS7 a A f -t i>m£i 3 *l X H £ i: . 

i-i? * y r l±m-fI$feS^ 5 Xft<D®.?tIfZ>k %> o t 

[5]-'; y-X*§g^-t-5 2o© K^Y A* $ |il-@$feS 

raoy y-xsij3T%&£-r-&fc&K®ffl<*ft*, 
c t ic & -s „ 







m~e&Z> 0 1 OCDS^^^«t5?|l{b^(standandiza 
tion factoryC&D. F7^Aig^$/cMI6 

[0 10 6] fe^lOCDtM^DAf^^ 

:r-£{*§l, ^^-tTfML (cancellation) ft 

[0107] fi£-?t\ zt>7?JT> h * F^^fctfg 



[0 1 0 8] 4FWH»^y*^XA*«ffl-r*i, */MiB 

?*>tc&>t<c2rWL<Dmte*> F^-Y^H««K:J:ot:S*a[«c 
*r*s^t?ft< t F^ «SR«tBtcj6 or 

tt©«*fc % F7-Y^H»#Wtta<Z>ifi^*3e»L, C 
[0 109]»C 4 H9*J<fc^H 1 0*#MLrBWBT 

C^iSW&ffitt^f^flc-r ^itO^f^ 7'^^t7 D - y^ 

[0 110]77*77'144^f>X^-Ht 4 V4)l> 



(23) 

43 

£A 1 4 6<D-OXd?>Xte£o:y a B 1 4 8cr>>f 
>X^>X«^L-1f • ^e- F ■ x-^x> Kcj;oT{f 

»Ke> • h yccHor^ti-eno^ io 

[0 1 1 1 ] If-F • ^-f-^»JflBlx-s>*>hW, x 
y'l 5 0r7^Jl/^A 146£:7^;l/£B148&c 

**i«ara^u fUffloitB«c««tr> ^y^hyw 

«rr&„ cft&ojRtyctt. tir»i,fcj:5K:. 
i 4 6 i i 4 8 0-ewn©b , > - 7 7 n yo# 

-XA*ttjHLTfrto*is. 20 

[0112] ±K««©R8**«*.4£, If- F • 
tw»x~>?,x> MS, jyfycc^o/c^-^ • v * 

7*-7?h*8ffin. CCD^«X^ >^1 5 2-C 

JlZ^y-— F • a— ^ • x-iy^> h#sR&o7d£l> 
5 ICTS. If- F • '<-fwH0»:t-^* 
> V\%T^ %i£§jky'&^f~ 4 (data intersection prope 
rty), h#P^if$R, *5<fcCflK«tT> • ? r * h >;*n 

[0113] A^J^ 

7^1 5 6rtf-- F ■ - ta • h 
tc<torf^sns. A^7t:> • >r 1 5 4& 

^>X15 4^/c««r#^> 0 A*bT 
>-^>^^>X154 ©ft$£#«aS14<0tfe2* $ tlX 40 

[0 1 1 4] mm&Tcf&f&tc&tc. tttt;e>--r>* 

>X 1 5 4<D^> F;U^:NtCreateFile3-;UCDc|a-cy^^ 
• >f>X£>X 1 5 8#CO<fc5*CLrm&;**l 



10-283199 

44 

[0115] f^rc-f* - h(DX£ ? *gBS^* 

9 C©F5-f'<K:»6*iS I RPfflCCX*^ 

cd F^-f>**J 1 occ**-f ->ys*rrc>a#>if9*>K: 
isdt, *<hr^M^4c<fc Altera*. ^>xta 
c<d^h&2, iftKft&tf. tB*fcr>- -y>x£> 

*tt I RP*jEO<f&SSU tefflTSfcaiXDiELliX* 

[0116] ^*^^>ysn^tf> • >x*> 
^«d*iEL< war Xttt?> • -r>x* 

(C<DWlr«, A*tf> • ^>X^>X 1 5 4) fr&M 

^oT, raao (Win s^^n/c) a^e"> • -/> 
x^?>x (ccD^rii, tB^;b*> • >r>x3?>x 1 5 
8) *ra^«c«#«KRori*<!M36***. 

©7^^^lO^C^x>f^>^$ntC^l 

»K«B^©a^«^ * ^ ^tBS^^y - *©BBIK:|« U 
[0117] T-<TOb'> • -/>x*>x*Btf#tt««: 
ccr^in^^fA^ MiSA^iffl^ 



(24) 

45 

[0 118] 7 ^*±ccmU3ft**ttfc:> • 

JE©SS»t:> • 7y ^ h Vtt, 16tT^KD44*a^ 
;l/7PCMt-f^ **8e^©2 2tD^7PC 
M*-7W *&£©. C>< ^©M&Sf*-* • 7*- 10 

8§ftfcT> • Vy >? V y^-tOf 5 -^ . httdf 

?2>C±&~CZ2> 0 HPg©g^f>- -f>x*>*ttt: 
[0119] iji— ©J* h y -AgiA#£/cteX h U - A 
3B»«HlWtf*>*i*^ hV-5> 4/IR«r«, I R P$lJ 
SKD^V^Fm USUI RP(i§7 

*«c<fcr>rfWS3h, #©?^;i/#^6ftT#ts<* 
ft, C07 ^W>£ 

©I RP*HWBS*i, AtH*v*-y*i«Sfffflr* 

i^ratswiLsntt^. *-w>±©«*tt7-/ 30 

3 ft 7 a to Z lffl©ffiB»«WJ53&SKS 

rttl>©». -ril'jrttl RP©fXf ^->3>(d 
estination) /c^ot C^ft^J: < % AtttfT?*-^ 
t*3--il/U I RP^©7^;^^Ci^ 

itt, ff±«»*»6©88«tf> • >**>*©»*# 
S^ffl I R P t*WKS*«0 7 -f Jl/ * 6*6* 4 J: 5 

■c, $fc«:®ffi©/ctf)K: I RP4ML/fc7 40 

[0120] tmmm*>-*)i> • k • 7 * ;i^©;$ 

60 ^ h A • T^7*j ©f> • ^>X£>XtK$l3g^ 
33ft4 I RPtef&iESft, SF7^^'*^^^h 

f&o 3 6tc, «e«JA*t:> • ?r * h y©5jiw» so 



1 0-2 8 3 1 9 9 

46 

*«£©«« t> ' 77 ^ f y©a«^D^-f «« 

[0121] W^f> ^>^jf>^ (JfcttlRPV 

>X£>X I RP-»*(sink)) *«t7 7 ^ 

;l/ • t^yx^ h^©#^»NtCreateFilen— Jl/O— JB 

£UT3i£ig£ft4 0 attttffiju^>F^«^^^u 

8) ©fv^X . ^y>^x^ h*, A*SKt> • -f > 

4 6teiiBa-r4fv*-fx • *zfv>*.>? bit. agcc^r 

S^r«7 a)izbi 4 8KMii-r47vW;* • 

x ^ h©^ft^6it»a^ft4x^ ^ass^y-** 

[0122] y I RPSrASHTai*. -€-©» 

-f^jrffl«cjBjesn&«««:^ri»*, irpx 

X£?*&£;s^-££#Mt-£a\ *4l*«-€-ft* 

ur*(D7 ^;u^© I RPXZ v ? • a^-^>3 >^c{4 
I^SciWot, «iffi^*^>±©^©y -/^^ 
ffl© I RP4W4o 

[0 12 3] ^ -YJl/^ • 3- R£ I RPX^ 9 ?ft<DX 
©a^-->3>^:?SHtU, Affl^iv^->?t^3-;H 
riRP^:, }§S3ft/cii0CC^©y AtoZ^mtCt 
«tf»otl»6. C©<fc5tc-T4<fc, «f 
^©&i&b*> • ^>X^>X^r^-T v <h'©^T * 
1 RPiRia^-^SS^ffi^TrffiffiTS 

^fg^-r-5c<h^r#^ 0 stoT, irp©«^i 
©/c^cc-eft-eft©^^^ • ^^^^ » ^ 

T4/c^©IoAttachDevnce©J: 5 ^ctl^AW^^^-v 1 
• r3-;l/^ffi^ft^C^ e 

[0124] aat^tcitt gib*> - >r>x^> 



(2 5) 

47 

Oi*4ttfflr85Cit*5. *©J:5tt4©40 10 

t, F • F'Jxri/^ F^rHUlfrc© 

[0125] jtfcs mi i tmi 2te<t^Bii 3&mm 
it, mi (setsaw) £02 (taa^*/-*^ • * 

— F ■ F^-f'<©'W u^;u • K2i> v ?m) icjfrtrvy 
^jKffccoiirBiwra. mi i«#^xie2ti 

• K • 7^;bjr(offiS««*^JSRT*«^ 20 
0 1 3CC7nT7P-^^- hcc* >?x 1 6 4 

[oi26] ?^x<D*mmigt&ftt>tiri,>z>m 1 2 <d 
flxo<t»&#**ft^ft©«ttfc-> • >r>x^>x43i 

[0 l 2 7] Xf^l 6 8*p6X£- F Ofc#>£, $U 
«lx-s^*>h l 7 0^r»^l 8 Ot, V-#-y 
^^172, fn>^-;1f . ? ^£ 1 7 4, 

• y 4)1>Z 1 7 2 • F^-Y^* 1 8 2 <h<£>fyj<D 

iiSft&cfc^K-r -fju* • ^>x*>x©m£ 
jvtKDJ^zzyzzm'fy r J • *7V>*>? y^<d 

[0128]^f9^184t ( U"-F ■ ^"-fw • x 

-is^vmh^y -i^zii 2s T^y'vyy- - y 
A)\,$\i \ s n^y ^^176, *5<to'if^> f • so 



0-2 83 1 99 
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U>^»;>^* 7 1 7 8&cM:£U «JRb*> • y 

*^<^^-h-r4*\ &mmt> • * f 

F3ft&r-£ • F, iIfg/<;*<D»f** 

• f • y jfrZte. mnu (09*. 

[0129] Xf77'l 8 4'CCDCCDcfc i 5^C-r^r(DM 
tf>Ccffiffl£ft&o If- F • ^-7^ • x-^:*> MS, 

[0130] F Sft*? 8 -* • FCC» 

F, ^yf-^i^'DAf^fiiMSi, {S^± 
CD (hypothetical)^ ^ ■ 7 ^rf^iS^^ C cb^r 

«tci^. y 4)1* * f^yZftmL <£ 

#vy*ftr8rtrz>£>>$c?z>±, ^tfpJtgr 

[0131] jEL^g^ttfR^^ori^cbXT'^^ 

i 8 erftgsnsi, A^7t*> • -f>^^>^%M 
^^-^ • x f y-A0y-7^e>i4)i< ^cgtn/cA 

SCim^ 0 Sot, SSK«, «ffrtcffflcSti/cA 
A > X ^ > X 4 ffijflt U /c (0 -C ^> ^ o 

[0132] co^$->«, -r^<rc7)A^t*> • > 



(26) 

49 

b*> • 4 >X£>X2&>e><D^> Fil'SPSftD? -/;U*±(D 

[0 1 3 3] Jl/-^0«*J©«»)ilLT«, A#b*>- 
-Y>X*>X 1 8 8ttX-?**:/l 9 Orf^JSRStli. fP 
ffl»ffi^6^>F^*g»aSlS<k, 1f-F • ^-f-f« 
9i-^x>F 1 7 0«*©^> F^NtCreateFile^ 
-;l/"C^y-^iUifflLt^f^l 94TfcH;f> 

rcn^rtf 5<t, *-?>F • • 7 -out? i 

7 8w. ^n^n>^i£-r^g^b-> • -y>^^>x i 8 20 

8 t 1 9 2*iiorsa*7 ^j^17 6 ccH3»w«:J8tt 

3*l& 0 3HHS6T«, NtCreateFile=J-M*3.-if • * 
— F • *^Y7>r#*iJ/rc#*AP I(DM»3-;l>© 
— SBil/T ? 7* (wrapped) J StiS. Cftte:J:9, 
^ p . . x -^^> FOjl— !f • FIB* 

■MB**— a— if • F A P I fcJIMiS*ac 4**T 

[0134]Xf-;7'196t, If- F • ^'-rw • * 

«3»WCC». T^T©«»*JfT*>tl, If- F • 
/*-^**Wx-s/x> h 1 7 0&*.x h y-Aftf-^ 

[0135] «_b<DJ:5K:LT % A*b> • ^>X#> 
X2 0 2liXf^7' 1 9 OTsK* 1 6 4ti^n/c 
^-^(D2HB©ai0igLr^3<i. (ft*, W^b> 

• >f>X3?>X2 0 4(iA*f> • ^>X^>X2 0 2 40 
CD^> FW^f^ 7* 1 9 4r^<om&cD— 

[0136] ^f^l 97"C, If- F • ^—fwfrJW 



1 0-2 8 3 1 9 9 

50 

*)ttt«a©»«t> • a >x*>x & 
a©fc»cDf ff ->**»ws*«©A*fcr> • ^>x* 
>x) *^6te«>r. > • -Y>x£>x (09 

^-^^^^CCj^OjitfafiKDW^b-^ • -Y> 
x*>x) CCBJJ«-&*rtBE*«W--*^ -*-F • 

«RI07^l/^j!fc(3:^yy^BIRP* 

«3*iSJ:5(c+»«:»|iar6ti4. 
[0137] jftfftc. If- F • 

1 9 8tf-^^iL/t^e>^7-y 7"2 0 o-e^j-r 

[0 13 8] Hu}£Ufc<fc5K:, ffl*b"> - -Y>X£>X 
©SfWRfcti, ^C^cS^$n/cA^b-> • A >x#> 

€>o C©:7WJb- r#flg«, m^b> . >r 

>X£>X(Dffr?)c;>>> F^^A^Jb> • >X£>Xk: 

[o 1 3 9 ] b^tMtfcmicmw-rzt, ctitcj:o, 
A#b*> • ox^>x^ti-r^ux • 

-z<DmuTt>K*zti. mmztx. tB^b>- >f>x 

[0 140] ^agS^^-^tt. ^ffilRPX 

tf^i, affi*^-' !f • FCCUtt<rfe, 

-I RP*$W»Ut < CW^„ 
[0 14 1] «±<DttW*i6ffl»3n-SJ:5CC, |5j~<D 

c<b^"C*s. £A;*H>**>*ttinJ— ^-Y^r* 
0, 7 -<;u^« iocD^-Y7*cDA^b>/c^*if^- h 

[0142] jM(DC<b*tf 5C<hfcpJfffi-C&& 0 



(27) 

51 

0, v * • 7 olZtem— A*Si^t'> • AlsA 

actecfcOf*-* • XF y- A£tt<cr& c<ta*njfcg 

ait^ fcSSR-* ^ <b *0«3l«fe 

[0143] K9^^^H*#*«lfefl[CC|»S'C*5*ay 

*-Xa (WU£, y'n^fw yy? K*£\ *$ 
iOV'O FH-&) 4, -f^XCD^l/y'zr <4T> b ' 7 10 
^ ;U * Kit F 3 & c <t tc o r i^-tt <t«ig{b** 

x7 • ycm>f #frhmm2tiz*>7-?<<T> h • ^ 
^^**»^<t cij&s-r**. zhttc, mm 

z>7 4)i>zm<Dttn.mm*$mtcft*>i}'- f • A*--?-./ 

[0 144] ;XCC. H14€r#«8Ur|»WrS<!:, H« 20 

tci><Dv$>*> 0 ekc**, &j£<D!im^>#-*> fkim 

C v < y 7 r CC Kit 3 ft & C £ < 1*1 D U *r - a > t? 40: 30 
S£tt&CDT\ r|5|3f(in place)J rtefflSn* <bl*tf 

[0145] ^>^l3>^-^> F2 1 0B^.;7 

- jt*V<D7 ls-J*&$t¥X2>t%mm2tl2>'*v7y 

fc*5, V-XMl3>^*>h2 1 4(*^EP2 1 
8tStJ: ^C:»^ . ;< 7 7t • Tn^r-^2 1 2£ 
#JH"i-*^*y r • 7a^-^#12 1 6^^>oW 
&„ 3 KS«iffln>si<-*>h 2 2 0«5?;l/ (ffi 

«!sffi=i>#-*> F 2 1 0 bm<DF]2 2 3tS1-<j;^C 50 



«fM¥ l 0 - 2 8 3 I 9 9 
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)?W-;7 7 • TU-Jr-^M^rfcoT 
[0 14 6] COSftftjBIWrB, V-^«ffl|3># 
-^>h2 1 4(^>^ 7 7 7 • 7P^r-^2 1 2 £ 
ttffllt^-;77'7U-A2 2 4a*a3r 4 ^>;7 

»St:6*ifcy U-A2 2 4atCtt. ^En2 2 6"C7^T 
<fc5CCy-X«ia:3>#-*> F 2 1 4^J:otf-^ 

wmttztiZo y-^*affl3>^-*>F«*4 
*>ft/cy u-A2 2 4 afc^-^isariitfci^rfir 

[0 147] C(DB$^r 4 >»^!3>^>h2 

1 4«&a*»iLT*5 0, **a<Dn> ba-)V&&W2 

2 8 -C^f J: 5 {Cf ftffila >?K-* > h 2 20^|# 
fflrr. *v7r • ra^-^#RS2 2 2<D#RftO**>>M» 

^^nri^i^cor, ^13>#-*>H2 2 0 

M-;7 TCOWO^Tfe, ^-^^r— ~n<D^v7 

9 7 7 ^RStS c t fcff toft^, fct, ^ 
Ml3>^^> F 2 2 0&*£cEP2 3 0rStJ:^ 

fj^r^n/c^'^y r • y u-A2 2 4 br risj 
[0 14 8] T^frttSfi/i^* y t • y u-AKHEiS 

^ntl^iCW, ^'•;77'7U-A2 2 4a, 7 1/ 
-A22 4fcu fccfc^'y U-A2 2 4 ctemC7 U-A 

3f£»r*4. ^EU2 3 l»K3r6hfc7 U-A^y- 
Xffil3>^^> F 2 1 4<b^fft«ia=3>#-*> F 
2 2 OCOHirSWSStiSCiftTnLT-C^., 
[0149] *«CC % ^ft«Hl3>#-*> H*«H1© 
n>Fd-;^, ^Ef72 3 2-C7K*TJ:^CC->>^®an 
F2 1 O^cgiSffi-To ftfc. S^ar3>Fa- 
^<h^Wl, (SJDyU-A^yU-A2 2 4 b<b2 2 4 

2>o COim^b. m^C^J:^^^ yU-A2 2 4a, 
yb-A2 24 b, W^'7l/-A2 24 cBl^nfe 
y-^«Hl=i>#-*>F2 1 4lCj:r>X&micmmx 

[0 15 0] ^>^»fc>^-*> F2 1 Ot*^-^ 
omWk^ht. ^E|J2 3 6 TtS-T J: ^CC^f^y-rrt 

©«ar»#7 2 2 4 c *«ttt*. is>zmm 

:2>tf-*> F 2 1 0ttSfA^7^«fflLtl^^ 

or, ^EP2 3 6«rttwccraj^or^>^^arj>7j<- 

*> F2 1 O&J&l/TteO. C(D7U-A»SI^«i 

t^ci ^s^*r ^ c t h pjtire* 

[01511S15B, ^>;77 - 7P^-£ • ^^7^- 
XA3&s±3$ur*/ctB5S!iR*-*^ • F • y 

T^ar<t*(D<t ^ccttawtcjiesns^s^i/Tt^ 
iuiii 5«*tcBit;y -f • ^y • F^ 
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M«F^'<£*©^tt*ft**i> :x-if-*:-F- 

*^7>F#F^-r ^4«fBir z> c t * nueccr & t 

ft£ ( I RP#*-*;l/ • F • K^-MKJ^Tft 
K^r*^>) 0 io 

[0152] V-«ia>^^> h 2 3 8CD>f>* 

^*>X2 42»*»L<frJ6U«:J:5«:. ffl^b"> • 
>x£>*2 4 o^eo^sfeo^^ccfpflsstiri^ 

* . 7^^2 4 6(JHj*ty • -Y>X£>X2 5 0 20 
[0153] tHSSiKSft/c*-*^ • F • h 

*»SWCCfe%. ffl*t:*> '-f>^$>^t07 4)1 
fttC©#MfflfflLtA'^7 • ? U-ACDfJ^Ti 30 

^:7£mK0/c-tf- F ■ 7 u -f*0tBlx-^x> hcD 

[0 15 4] A*f> • >f >X£>X24 8±CCJgJsR3 40 
hfc^?7 7 • tp^-£ • ^ t -4 )\s • * 

• >f >X£>.X2 4 OffirtyVT • 5 2£ 

i 5 tc^R-r^-c^, ^^t»;(D^ u— a^s^t-a • ^^e 

»;2 5 6^%«i^re.nti^o 

[0155] y 4)\s2tem*<Djjmv&Kmmx*z><D 

X\ ;<?77 • TP^-£tt* fflSSntl^«^r* 
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?n^ 0 3 fy *)l h(D->xf A • y^:?7 * T 
[0156] If — F • ^-f^Waia:-^*>Ht IS 

^7Stt(cocitii$it^e>, mf&(oy jjvzm 
^clu tcjs^ *tf 9 c <t a* oj^cc a o r c ^ a a* % *hss<d 

[ 0 15 7] If— F • • ^^T> h£/ctefrj 

jlx^^F^-^ • F * y 4 frZCD^MM 

A£ffifftT£©CC2tU m*fcT> • ^>X^r>^«A* 
fcT> • -f >^ * >^«cBlt5i«J6^ y 7 r WSt ^ 

»^-r<oa**A*tr> • ^>^*>^iU*tr> - 
[0 15 8] A'-,7 7fOr<Dsmt -r^<r©3>^ 

m&te, m<Dm&±mc£5t£2£x&tj:yy&xmf&? 

&C£&*jmx$>2> 0 mtf. ^n/^-f © 

P^-r «*fe > F {b3 n^:«ffl% ^>of- ^^itt? 
^>£o ^>£CHi, ^oAf ^ft^^a^cD^P^^-r * 
b-iCti>nimx$>t), *r<Dm&. 

fo&mL^&^xte£K>mmnxfoz> 0 ^ 
tcMci>mmxzz> 0 
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[oi59] uroymtf-zmmc. tititmmm-? *<D£5te$i^i3&$ti2>fr<DM&$>-££ft-c^z>. 

a^'f i l/tfl*An5 C i Al-Ct 47 I/ - 5 [0 16 0] 

(S3) 











^ X x A tC 3 ft ■? A" -f 7. ± <D * V K 

rA-f X 1 1- Ttt, 3* y K-f V ♦ rt — K . 
C © J; ^ ft PCMCIA* — Kfcfcra***, 

T a - * t & •& c t & Jg£ T £ . 

m © T a y - ? gij o ^ -c n S t £ If ffi * ti i> 
©* J jI1ttT';&-5>. 7 -r )\s?i>m%.<D7 v— a% 
SEET4#E**fct**Uf » ^7 V • X h 'J — A • 
T a — ^ «?> ifijjo 7 U— A £ §lj ^ T -5 £ t ft < 
*®7 u— A.%y^ v • x h — a • 7 -< A^tc 



[0161] [^8] 
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mk^y • -i (AfltfcttS'V*) <z> 
7uy-fWfrznmx^z>o Sfrtbtti 

3&M r - * * § it Aft £ «fc 9 K SiJ £ T ft T 
tti. c:<z>®{***ia£S4-iTiAtti,\t v A2> 

t±Vam j rJUA±cD*y#- HRAMifcti 
ffiOft^CD^Is^tCprtJC-JW-C^iS/X^A • 

y*';*ii0t-r«. 

7U— AfS-^tt : u(USttll^^^XK 'J — A • 

7 ;^«S7 u-AWr-^S^tt^imft 
tic fc*fflatfv • A yx-f VX**S*bT^-5 

^L.Ti>4C i:^.ltWti* J . tttatf-rvHi 
:7U-A*$QgT*. 







[0162] 



C*93 
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[0 16 3] C<D»»©Sj|«fte>t4a»3tiSJ:5 

[0164]7^l/^ - ^^gfk&stJ-- K • ^-f"-/ 
• K • A'*;7 7 • 7D^-^W^ 

JBU ;<?7t • 

7P^$^gT7 7 ^ • H*S»fcT> • 

[0165] #l^gSAn«. SMf> • -r>x£>xc9 

^fe, NtCreateFilen-;l/^1f- K ♦ -r ft'fflx- 

[0166]^y77-7O^-if - >f >X£>XteC 



^©AP I iMLXiHtJ^> • -Y>X£>X • ^> F 

[oi68] ^x^Accfflr&SftT^&r^ h - r 

^X • F^^©fctf>6CS'Xf-A • y^rV • ;<?77 • 

40 h T * c 4 cc J: -oXt&m^ ? 7 r so^r 

tt. 7 -f^iMRtWtt. (l)jj!*^?7r*StSft 
B*CCj£iL43- K*9MHU (2) 7=7 * TP 

*f?^xi*<Dfri8u9&ifiy -< ^zzmLxm.ztiz> 

50 Zf)\sXpZ$<&£>$tl. f7*)lb • TD^r-^O 



(32) 

61 

F 5 <^<DT 2 -fe *#-C# & J: 5 OCT &o 

[0 1 7 0 ] Try + frh • rtvVy • TO^-^(m 

[0171] F • */cBfte<D7vwxtt: 

Jl/£B, rtvVr - • -furtf-iM io 

- 7^—7* • x-^x > FB7 -y ;Udr£rC v y ~y - TP 
[0172] ^P^^^r^b* * 20 

IBTtf-Ohiyr Jfr • n>^*x h • xyrocgo 
t<,>£. coT^aijSfcB* iRP^i^oiRP^ 

l>Tf^<D>^> F^iM&/c#><D'r a X^v ? • 7*-:/ 

r^<D#R3. *ST7U-3>yRft<2F-*« 30 

1 RPCc<tot:*{istifcfe©) ©*a-at*3W* 

[0 17 3] CCrBi*Shri>4SOt<D»lll©^y7 
7|l3T^^XA^]^l,>f>3?7x-X, OS 

a-1f • *-F - *7-rr> r£IEL<ffl{iT£fcg>fc: 

U-t* ^ >^ • ACD^r j 7s • U^;b (if — 

(D^x^^<D^n-fe*>ifr^a^^ - 7^ h^n-Si 

**»^«. ■■fc^-^b • ^^x-XW- h 

^fflUt (DPC*MP*Ct) ^7*-v>X*f6l±-r 
SCim-So CtiCC<t»3. I RPfcAHtfr?*— 
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[0174] IRP<-X0O^7x-X(iIRP* 

PCC«S (pending) I R P470^*- 

[0175]^»7t«7 U-A^7P^r-^CC*JfflRj 

*>ofcI RPBCODS^+^-CCg^tl, MSO/ctfHC 
[0 17 6] tt«R|«=i-U- y-^tfltflW-Sf-f 

tiarB*3WW«[3-iU (ctittDPCWctf^c 

C©ia4S^ii, — • F * U ^xx 
[0177] fiKrtv* • • -<>^^x 

• 7-ff*A(worker item) ^rX^^^-^UfeW 

^rffifflT^) CiBDPC U^«c*4 pJfi6tt*SiftTi 
fth-ChZ> a S*WtC«, I RP©^p%tf?»J»7-^ • 

[0178] CCrp^ltt^A^77 • 7 

n-ir — £ • y^^XA^M D L (memory descriptor li 

st: y*yiBJ6-fyx h) <h-ificcffiffl-r^c7)cc®Lr 

OSCD-C, ^^^TlBO(EaH»*S6«:l!ijar*Ci3&s 
-cars. c©<fc5ttjliSK:J:*<k. NT^u-f^> 

[0179] ^CCC % a 1 6^#MLT|ft0JT€><b, 

oc b mf^ u fc^< *^ ^ r su^ r ^ io ~ x a & mm? & m 1 
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-7 8(2'<9 7r • 7 U-ASr-y-? 

> K • F • 2 5 8jfr6«ST4iftfi***0, 
r r n>^U ylf - 7 74«f-^«:f^^ • 
F9-f^2 6 2^6iS««:SWffi*C<!:*sr*-S5xX^ 

[0 1 8 0] fflWx--i>*>h 1 7 0 tt?y *7 10 

5/** h-cstsn, A*t*> • -r>x£>x i 8 8±cc 

JRfiS3*l*^v7r • TO**-* 2 6 4ftffJS6^S. 
v7 y * 64B^-;7t • 7 U-A£U* 3 

>F • F • -**y 2 5 8*>6»3r, ;<77r • r 

P^r-^2 6 4-vO#I«Sfi2 6 6^?n/cttl^t* 

> • -Y>X£>X2 0 4CCB8sestiS. C<0#HJtt^9 
7y • TCJ^-^2 644iT77^^ • XT**?**? h 

FJUitCth • 7 U-A^SI 

SrSfefctc^n^^Uylf • 7 1 7 4&c£oT 

[0 1 8 1 ] (II Get ^-;7t«7P^268 
fc?y-Ol/ • 9 hCCj;oT*ti*ft, Aftt">- 

U>r-^2 6 8 fct^X^A • ^-=ey 2 6 OCDU^r^:^ 
S-T^o W»x-^*>F 1 7 0tt^?7r • 7a^- 
#2 6 8*ffil«Lfc*£. KM 2 7 O^tST <fc5fc-€-<D 

fe, 7 7 • 7P^r-^2 6 8 A • >^'J 2 

6 Omo^yy y • 7 U-A<DSH^r£iI^U 5 s -* 

^^^2 6 2^6*ccc|E3lrtSJ:5tc-rs Q 30 
[0182] *Jffllx-^*>F«a^tf> • ^>X£> 
X 1 9 20'*? 7r * 7n^^#i©f(C^Jl/ 
fi) £Aft, BBafflbWT*>ti«Ci**L, 
rt^ 1 7 6tttJ-*> F '*-F-^y25 8rtCDgJt# 

^m^®fflU/c^<tTf r -^^:^^>F • i3— F • ^ 
*y 2 5 8tCKT 0 W8Px-J/*> h^-;77 

[0 18 3]^{C, il7(D7P-^f hiSl6© 40 

trbnyt*ir|Hf Sft, x f y-AflBKQ iSiA^ti 
3HJ®Jx-^*> h l 7 0*»6U-y • 7 1 7 2K 

[0 l 8 4 3 «tntc, y-#- 7 ^;L/£ 1 7 2»Xf 
7*2 7 2t, y"^>7'Vv^ *7 -OXs** 1 7 40^y7 

7 '70^-^2 6 8^fflltVXfA • ^^tyce? 
u-A^fi|^r^ 0 y-^- 7 l 7 2©tU^t> 

• ^ >X £ >X 2 0 8 A > 2 7 0 J: 5 (C % ->< 50 
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v 7 y • 7P^-^2 6 8 &^*?7y 4 ^ • 2 
h^(D>^> FJU*Stt8l£<DT?, iStC7^-feXUt^ 
77t- 7P^^2 6 8*Wf9T*C4*«T*4. 
[0185] 77^1/ - y-jf • 7 Ah* 1 72lt>X 
f*A • y^ey 2 6 OftCDHi^CD^ "7 y • 7 U-A'MD 
T^-feX«i^f#^<h, Xf ^7*2 7 6T^EP2 7 4-CS 
T<fc^Cc, r ^ X#2 6 2^6cDf r -^>&>' U-A&CA 

£ 1 7 2«f-^^^rA ^^'J26 0KU#%i&tf 

[0186] #C*C, 77^Jl" 'J-^-7^^172 
^r>^ 7*2 7 8tfp>7'l/ 7t'7^;l/^174^ 

ttl RPU-3tNTAttl*v*-SPt(Ci8Sn5. x 
r^'2 8 0r, fayT'U^f -7^^174^ 
Ta^r-£2 6 4*ffifflL/rif^> F • */- F 
•^^y2 5 8©7U-A^HaUo fP>7'l/^ 
• 7 ^Jl^ 1 7 4#^»7t - 7P^-^2 6 4 4*Do 
rtiiOl*. *C^<D^>FJl/#Hfc&tf> • -Y>X#> 
X2 0 4CC*tO~CX F73nt^/c/c^So 
[0 18 7] f3>7*l/^f -7^;^17 Ate?-** 
*f*5RU StfP2 8 4^*J:5{C*<D? s -ir*, If ^7 

> F • f • -rf*y 2 5 8fc«mtcfjsr e>n/c^ lx 
f • f • ^^y^^so^ren/c^u-A^^xr 

^ 0 CCD^*J/< v7 y 7 \s- Atb^vgCCft^CD«, 
[0 188]lg^Ci», ^-£3:— -fi<D^v 7yfr 

hm5<»><v7y^nv£?z>t%, fei^n^f-^i 

jft^tltf, $liA'777Mfif^X^-7 

^y>^*fc«^3Wff*>na3CPCcf£;Dr 4 ^<tt€>J8 

[0189] fPVT"!/^ • 7 -f Jl/* 1 7 4«^^ 
U-ACDf*SB^^^<b, N TAttl^^^ -is* (Omfe 

zmmorz h y-A#j£#*a&*:7 ^^17 6cc?b 

To 7 ^;^17 etiX-r ^*2 8 8W h y-A 
•MIRP^K^ KSlf ^> F ■ 27- F • 

y 2 5 sr^-^^gi^fMaT^o ccDjajm^ffl^^- 

[0190] xA*^ ^^17 

if r>> F ■ u>y y >^ • 7 ^;u#cc?gi*ft£ 0 c<o» 
^i), y-A*i&# 1 RP«r-y-^>F • u>^y > 

^•7^1/^17 S^dSjUS-a-S^^xXAfciSftS^ ^ 
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[0191] ^^>K • u>^'J>y- 7^ 

* 1 7 8tt*^*:/2 9 2 -CX F y-ASi&# I RP* 
SWffiD* F • F • y^'J 2 5 8&c#:£-r& 

*C1^> F ■ F • ^- F^*T*«iaiT4. CCDB$ 

,£-c\ «Succsi|^r6nr^/c^^> f • *- F • 

£ e rtvVy • 7U~A#{^pJfter*4C<htt7*3> 

:/u*u- • l 7 4cc*ne>3ft<5<DT\ f^ccg 

*»ftT(,>£X F 'J-AWiStHWl/tf-jHrjHI 
U a^l8flc3hfcA-;7 7 -^u-AKAftSc 

[0 19 2] JMc. SI 8£#MLT!ftHJT£<f:, Htt 

* ■ X F 'j-ASrVX* ^O^ - 20 
U>y^2 9 Btt*EP3 0 0-C^*TJ:^CC, 7^ :/ • * 

-^-r* • xf y-A2 9 8*sftLr«ai-rs. 

tw:* • U>^72 9 6 #7*-* • -fr>:/JU*SrtlMc« 

^tc^ur, co«#*^En3 0 4r^-rj:5cc 
^> F<t or*nS;;*ft& 0 

[0 1 9 3 ] mW&C, tff*:* • U>£^3 0 6ttSaEP3 
1 0tSt<i:^C7>f^' fcf^tf • X F »;-A3 0 8£ 30 
giLTiltl). ^•U>^73 0 6WotJH 

JftSti, ^E[J3 1 4t?^ct ; 5CC^:^^3 1 2&tj*6 
ft, ^Ctf-^-ty^M^^-^iL-Cftll^n 

[0 1 9 4] ftSlflJiLT, 

A3 08i7>f^' • X F 'J-A2 9 8«PJ 

U> F&^'U-fe*^-^* >£Wl-r£/ctf)fc:, Xb- 
#3 0 2 <t^*3 1 2 T* U> # V > ^2ft& <t £ fcf^ 40 

4zf • fir:* '^F'J-A308<!:7^y-*-f^ 

* X F y-A2 9 8©A^W|g»J^b3ft&^ft«3tt6tt 

[0195] Cft*tT&9fc«>K:te, —-fi<D?-Z • X 

f y-A^v^^iasji^ccjstRSfts. ccm 

^7X^ -f-^^h y ~A±&*J:5fcffiRWK:« 
jRSftrc^. ttfc* 7>f^ • tr^ • x h »;-A3 o 

8#vx*£0TiaiR3ft5C£fc, ffecDT*- £ ♦ X F 50 
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y-A*J^ir<toraBR3ft*ci«>pi«r*o. * 

ft4&tf*fc<m*tt«: S6CC V 7 • ^5 

7 <&j*awc • x F y-A*c»--3<fl.3B*u* 

^^ffllt, 7^^'t-f^-^hy-A298 
<t5>(:/- tr^*- X F'J-A3 0 8 4ftteHttfM-* 
c<h#plt6-c#>& 0 fiE^r, 0t^cdvx£ • ? * • 

^^?^XA3 1 6Tte\ HJ«r • U>#^2 

9 6iM«ft, 88*113 1 • U>#^£*gtfft 

•^3i6 ^6raJw^fct(Wll*tt^6*>o#^^r^«■r s 

[0196]^^^p^316(t fw*- 
l/>^729 6-C©MlU- hSr^Vv^^ • ^-fw 
t-Xh U-A2 9 8<DU- F iK^ri/tfeCC^-f 5 
^•l/>^729 6CC<fcoTftJBT£C±fc"^|ffir* 
£0 «l>»*.ft«, 7>f^-^f^^h'J-A2 
9 8te, 3IJ<D^Xt-A±<Dtw V>$Mt7 4)\/%/*TA 
i?#Ml^- \rV*T%:£<DM<Dmm^>#-*> FK 
ctoT«I^tt^ 0 t-f^^F'J-A«f^ 

is$Mty >^i/^e«lfiJE3ftrc^(7)-c. cfte>oit 
y ^JM&ffl:a><F--*> h©^- F^xT 
fcJ:Cf*^U-*tc!(65U-F"Cft6»ftri^. *y^ 
:r;WMn>#-*> F**^-f ^ . *-fw * . ^ F y 

-A298^(HM^i^0, C U~ FCoriqina 
tion rate)i^^-f ^ • X—^j * • X h 'J-A2 9 8© 

[0197] t"-^ • x f y-AccAori^if >^ji/ 

>y';U- ^;l/-^«:^>rAX^>^*BSIiSrJWScir 
^^Itt^^ca^^ Witt r-d? • ^ 

t-f-^-Cttl/2 2, 0 5 0#^{4) r^1f>^;U 

*i*ci3&ir*s 0 $6&c, £^ax£>:/#su># 
— >3>B#Pa^TJ:^«:'rft5i, #So-^>^u^ii 

d? • x h y-ArtcDfigtcS-^i^r^^A^n^ ^€rf^ 
[0 1 9 8 ] «±^e»aj»^ft^cfc^tc, RpiaayttiT 5 
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-* • =F + y^ocS-^r, t*-* • x F y-Acz^k: 

rcpffir (discontinuities) J ^^-f-f y^tSCi^ 
< £lr>ofcc£3fc: x X F y-Afl^OX F y-A<DitHC 

oci#r#**Wfc**. 10 
[0199] v^^.^a^31 6*»6f#6*i&£>& 
BSiBffittf*-* • if >:/;kdx h y-ACcAoTi>&B$ 

(4a**uri^©rfl[KB$rar^>o, corns* ?x 
^•xh y-Att^-f ^-fw* • x f y-A2 9 

8£0T«tl?3*iTlr>*. 3e>fc\ ^S#*~tw:*- 

•^D^3i6tattsna, f-^-xFy-Afl 
©wnh-j-Aflw© ra^j interns ti. s 20 

<£ffi It «flH©ii** h ^ * * > & C <t (C <fc oT , 

. x h y ~A<fcRjt83i*4c&**r#&. 

[0 20 0 ] h'ft • 1/>^7306^7X^ • 

• 9Kl *j ZQyft^ 4? • rt-f^:* • X h 9 

* • u>y^2 9 erffiasnsBs^^iitf 
( r^(ticks) j ) tiot. ff±*3ft««cjhtt«* 

[0 20 1 ] JH±3^6«»Sti*J:5K:. b'ft'^F 
V-A3 0 8fc, RMMbiBH»«cU-h«E**tf 9«iW 
m-XF'J-A3 0 8(Dl/-M^J7>f 
^ • • X F 'J-A2 9 Sfcitftf-TW * • 

U>#^2 9 6t f cmLXMTX*mM?Z<DtmCJi&~e 

^- f$*t • c&hNfbb) tcg^x^ 

[0202]v*$r-*G**31 6 *>6f#e>*i*fc<D 
ilt, *-f : ^-U>?72 9 6flfi!)i8ift^ 

7^*- U>^72 9 6^7^7 % ^ftlrt-f^ 
cfcO*iBJSB$rai (*-f s -f * • L/>^72 9 6-C(DHBR© 50 
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tmu- h^-r) omfri'- 

[0 20 3] U- F&^W:. X h y-A • t*-***- 
u- F3&s^nH2^-9-<D«liarFtc 

U #B£U- F^^*n^7lf(D$fJlrF^g^nri^j:^ 

x f u-acdc itAO, tu^^utju^^ A-c^ffls 
Mc^^ti^J:^Why-A^6 0 ~>4rf**r 

• X h y-ACDWiOTW:. #<D<J: 5 
i)K Cft6teRJE3ft*t>©rtttti^ x F y 

-A#n~#;u£/cy:*7 h7-^li^ityr;i/ 
^Arissw-SJ^&'JT^*-* a • ? -f - f\ yr 
;^>fA«7^Fr$)^^ ^^hy-^^btl 
6*iT£fcX Fr»#7*-*r*£j&>CC|HiStt< % XF 

<j-a*** F«7-^«JK*>6Sfi"rscia^3&i* 
x h y-A»*©5fi &hhsk:«i 

[020 4] 7^*yf-/7 - Xh y-AA«4fS 
KHiT*«cC»tta, t-f^*l/>y7296^ 
olldinq)J i^«nStSSS*^C0, MK^^f-^ 

*3fe^c<fccctti. ftid>, 7>f^-^-f^-xhy 

-ArtM'-fw * • U>^72 96J:HfiU-hrf 

M1--<#f-^^<^^(!:C^f-^ rtSr^(starv 
ation)J Wffi^fiCSCitcaa,, i'^^OtB^fe, * 
-7^^^ • U>^^296rcDU>^y>^ - U-h 

7>f^'t-f^^hy-A2 9 8WO«fffl-f 
>»M-;utMBr*Stiri»*«*U- FCC-gcT<5<fc 
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Title of the Invention 



Method and Computer Program Product for 
Synchronizing the Processing of Multiple Data Streams 
and Matching Disparate Processing Rates 
Using a Standardized Clock Mechanism 

2 . CLains 

1 . A method for interconnecting software drivers to allow efficient kernel mode 
processing of data and having a standardized way of providing timing information for 
synchronizing different data streams ot rate matching different hardware clocks, the method 
comprising the steps of: 

opening one or more kernel mode drivers; 

forming one or more connection pin instances for connecting the drivers, each 
connection pin instance hierarchically related to one of said one or more drivers and 
used for data transmission between said one or more drivers; 

creating one or more clock mechanisms for rate matching and stream 
synchronization, each clock hierarchically related to one of said one or more 
connection pin instances and providing a data stream time and a physical time based 
on an underlying hardware oscillator; and 

interconnecting said one or more connection pin instances so as la provide 
a continuous data flow path through the one or more drivers residing in kernel 
mode, said clocks making timing information available for rate matching between 
different clock oscillations and for synchronizing different data streams. 
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2. A method as recited in claim 1 wherein each connection pin instance is 
represented by a file object and the hierarchal relationship is created by specifying the related 
driver, the driver referenced as a file object of an I/O device available on the system, during 
connection pin instance file object creation and each clock mechanism is represented by a 
file object and the hierarchal relationship with a connection pin instance is created by 
specifying the related connection pin instance file object as a parent during the clock file 
object creation. 

3. A method as recited in claim I further comprising the step of querying each 
of said connection pin instances on each of said one or more drivers to determine the 
availability of clock mechanisms in order to determine whether to instantiate a clock 
mechanism for synchronization or rate matching on a particular connection pin instance prior 
to interconnecting said one or more connection pin instances. 

4. A method as recited in claim I wherein at least one of said one or more 
connection pin instances on said one or more drivers support at least one predefined set of 
properties, set of methods, and set of events to indicate to a third party component the 
availability of a clock mechanism on the at least one connection pin instance and for 
allowing a third party component to form said clock mechanism on respective connection 
pin instances and the clocks support at least one predefined set of properties, set of methods, 
and set of events to control said respective clocks as determined by the third party 
component. 
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5. A computer program product comprising: 

a computer usable medium having computer readable program code means 
embodied in said medium for providing a standardized interconnection mechanism 
to other components in kernel mode, said computer readable program code means 
comprising; 

program code means for forming connection pin instances, said 
connection pin instances used for transmitting data to or from other 
components in kernel mode and capable of interconnection with other 
connection pin instances located on other components; and 

program code means for forming clock mechanisms on some of said 
connection pin instances, said clock mechanisms providing a positional time 
value that reflects a current position in a stream of data having time interval 
information associated therewith. 

6. A computer program product as recited in claim 5 wherein the clock 
mechanism further provides a correlated time value that comprises a reference time value 
based on a common hardware oscillator available to multiple components and said positional 
time value in an atomic operation so as to allow other components to perform time 
translations using said common hardware oscillator as a reference. 

7. A computer program product as recited in claim 5 wherein the clock 
mechanism further provides a correlated time value that comprises a reference time value 
based on a common hardware oscillator available to multiple components and designated 
time value in an atomic operation so as to allow other components to perform time 
translations using said common hardware oscillator as a reference. 
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8. A computer program product as recited in claim 5 wherein the clock 
mechanism further provides a physical time value thai is based on a hardware oscillator. 

9. A computer program product as recited in claim 8 wherein the clock 
mechanism further provides a correlated physical time value that comprises a reference time 
value based on a common hardware oscillator available to multiple components and said 
physical time value in anatomic operation so as to allow other components to perform time 
translations using said common hardware oscillator as a reference. 

10. A computer program product as recited in claim 5 wherein said positional 
time value is accessed by other components as part of a property set implemented in said 
program code means. 

11. A computer program product as recited in claim 5 wherein the clock 
mechanism further provides notifications of relevant events to other components, 

12. A computer program product as recited in claim 5 wheTein the clock 
mechanism further provides notifications based upon said positional lime of relevant events 
to other components so as to provide useful synchronization points. 
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13. A method for synchronizing multiple streams of data comprising the steps of: 
creating a master dock reference based on a reference data stream of data 
samples, each data sampie having time interval information indicating position 
within the reference data stream, the master clock reference created from the time 
interval information; 

processing one or more other data streams based on the master clock 
reference so that the other data streams are synchronized with the processing of the 
reference data stream. 

14. A method as recited in claim 13 wherein the master clock reference is 
accessed as part of a property set. 

15. A method as recited in claim 13 wherein the master clock reference is part of 
a selectively instantiate clock mechanism that is created in response to a request by a third 
party agent. 

16. A method as recited in claim J3 wherein the time interval information is 
provided by timestamp information on the media samples. 



17. A method as recited in claim 13 wherein the time interval information is 
provided media stream convention. 
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18. A method for rate matching a data stream processed through a processing 
component to a hardware processor comprising the steps of: 

providing, to the processing component, a physical time reference based on 
the hardware oscillations of a hardware processor; and 

adjusting the rate of processing at the processing component of a data stream 
to match the hardware processor processing rate based on the physical time 
reference, the data stream comprised of samples having time interval information 
associated therewith. 

19. A method as recited in claim 18 wherein the physical time reference is 
accessed as part of a property set. 

20. A method as recited in claim 1 8 wherein the physical time reference is part 
of a selectively instantiate clock mechanism that is created in response to a request by a 
third party agent. 

2L A method as recited in claim 18 wherein the time interval information is 
provided by timestamp information on the data samples. 



22. A method as recited in claim 18 wherein the time interval information is 
provided data stream convention. 
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23. A method for synchronizing multiple media streams comprising the steps of: 
creating a master clock reference based on a reference media stream of media 
samples, each sample having timestamp information indicating position wiihin the 
reference media stream, the master clock reference created from the timestamp 
information; 

processing one or more other media streams based on the master clock 
reference so that the other media streams are synchronized with the processing of 
the reference media stream. 



24. A method as recited in claim 23 wherein the master clock reference is 
accessed as part of a property set. 

25. A method as recited in claim 23 wherein the master clock reference is part of 
a selectively instantiable clock mechanism that is created in response to a request by a third 
party agent. 
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26. A method for rate matching a media stream processed through a processing 
component to a hardware renderer comprising the steps of: 

providing, to the processing component, a physical time reference based on 
the hardware oscillations of a hardware Tenderer; and 

adjusting the rate of processing at the processing component of a media 
stream to match the hardware Tenderer processing rate based on the physical time 
reference, the media stream comprised of media samples having time interval 
information associated therewith, 

27. A method as recited in claim 26 wherein the physical time reference is 
accessed as part of a property set. 

28. A method as recited in claim 26 wherein the physical time reference is part 
of a selectively instantiate clock mechanism that is created in response to a request by a 
third party agent. 

29. A method as recited in claim 26 wherein the time interval information is 
provided by timestamp information on the media samples. 

30. A method as recited in claim 26 wherein the time interval information is 
provided media stream convention. 

31 A method for translating a positional time value in a first component based 
on a positional time value in a second component and utilizing a common hardware oscillator 
comprising the steps of: 

receiving, at a first component from a second component, a correlated time 
value in a single operation, the correlated time value comprising a positional time 
value based on the position within a data stream having time interval information 
associated therewith and a common time value based a commou hardware oscillator; 

querying, by the first component, the current value of the common hardware 
oscillator; and 

translating, by the first component, the positional time value in the first 
component based on the current value of the common hardware oscillator and the 
correlated time value received from the second component. 
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32. A method for translating a designated time value in a first component based 
on a designated time value in a second component and utilizing a common hardware 
oscillator comprising the steps of: 

receiving, at a first component from a second component, a correlated time 
vaJue in a single operation, the correlated time value comprising a designated time 
value and a common time value based a common hardware oscillator; 

querying, by the first component, the current value of the common hardware 
oscillator; and 

translating, by the first component, the designated time value in the first 
component based on the current value of the common hardware oscillator and the 
correlated time value received from the second component. 

33. A method as recited in claim 32 wherein the designated time value is a 
positional time value based on the position within a data stream having time interval 
information associated therewith. 

34, A method as recited ill claim 32 wherein the designated time value is a 
physical time value based upon a hardware oscillator. 



(65) ftmW- 1 0- 2 8 3 1 9 9 

35. A kernel mode data processing system allowing synchronous processing of 
two data streams comprising: 

a first data source; 

a first plurality of kernel mode data processing components including an 
originating component and a terminating component, the originating component 
reading data samples of a first data stream originating from the first data source, and 
at least one of the processing components having a master clock component 
associated therewith; 

a second data source; 

a second plurality of kernel mode data processing components including an 
originating component and a terminating component, the originating component 
reading data samples of a second data stream originating from the second data 
source, and at least one of the processing components synchronizing processing 
based on a master clock component associated with a processing component found 
in the first plurality of processing components; and 

kernel mode component interconnections between the data processing 
components of respective first and second plurality of kernel mode data processing 
components to route the processing of respective data samples from the respective 
originating component to the respective terminating component. 
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36. A kernel mode media rendering system allowing syndironous processing of 
two media streams comprising: 
a first media source; 

a first plurality of kernel mode media processing components including an 
originating component and a terminating component; 

the originating component reading media samples of a first media 
stream from the first media source; 

the terminating component rendering said first media stream; 

each media processing component having connection pin. instances 
for passing first media samples between media processing components; and 

at least one pin instance having a master clock component associated 
therewith; 

a second media source; 

a second plurality of kerne) mode media processing components including an 
originating component and a terminating component; 

the originating component reading media samples of a second media 
stream from the second media source; 

the terminating component rendering said second media stream; 

each media processing component having connection pin instances 
for passing second media samples between media processing components; 
and 

at least one pin instance synchronizing processing of the second 
media stream with processing of the first media stream by using the master 
clock component associated with one of the pin instances of a processing 
component found in the first plurality of processing components; and 



kernel mode component interconnections between the respective media 
processing components found in respective first and second plurality of kernel 
mode processing components created using the respective connection pin instances 
to route processing control of the respective media samples from the respective 
originating component to the respective terminating component. 
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3. Detailed Description of the Invention 
1- The Field of the Invention 

The field of the present invention is methods and computer program products 
relating to timing issues in multimedia applications, such as stream synchronization and rate 
matching between a generating clock and a rendering clock. The present invention also 
relates to standardized timing mechanisms in software drivers. More specifically, the present 
invention is directed to methods and computer program products for providing timing and 
clock mechanisms used by multiple entities, such as interconnected software drivers, in a 
standardized fashion so that multiple streams of processed data may be synchronized and 
rates between different hardware clocks may be matched. 
2. Present StMEvf thff Art 

When processing multimedia data, such as digitized sound and video data, it is 
common to process a continuous stream of media samples. The data stream also has time 
interval information associated with the data either by convention or by timestamp 
information. Timestamp information will tell a processing component when a particular 
sample is to be processed or can be used as a tracking reference to assure processing is 
progressing at the rate specified between ornestamps. 

Data organization, format, and convention can also convey time interval information 
used during processing. For example, a file of video data in a 30 frames per second (tps) 
format and convention will be understood by a processing component, having knowledge of 
the convention and format, that each sample is processed at 1/30 of a second intervals. 

One problem encountered in processing media streams is that of synchronizing two 
or more streams so that they are rendered together. For example, an audio stream 
representing a soundtrack and an accompanying stream of video data need to be 
synchronized in order to produce a coherent multimedia presentation. 
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One method of synchronization utilizes a clock mechanism having a value that is 
based on a physical or hardware clock such as the PC clock. A data stream processor may 
query the value of this clock mechanism in order to synchronize processing or rendering of 
the stream. While synchroniaalion occurs with respect to the hardware clock, it does not 
necessarily synchronize two different streams due lo processing events and delays that may 
occur independently on each stream. 

Another method of synchionization utilizes a clock mechanism that provides a time 
value tliat is based on presentation timestamp values on a "master" data stream while the 
stream is being processed. When the master stream is paused or otherwise stopped, the time 
value alternates to be based off a physical or hardware clock such as the PC clock. While 
the processor of a second data or media stream may query the value of this clock mechanism, 
further facility is provided so that the clock mechanism will notify the processor at a specific 
time on the physical clock or in certain intervals (e.g., every 10 ms), again based on the 
physical clock. 

Since event notification is based on the physical clock, such events may continue 
to occur even when processing is paused or stopped on the master data or media stream. 
Notifications may be temporarily disabled but at the cost of extra processing overhead and 
increased processing code complexity. Even when notifications are specifically disabled, 
timing circumstances may be such that a notification still is erroneously made. 

Another problem associated with processing media or data streams is that of 
matching the rates of different hardware clocks associated with the media stream. One 
bard ware clock will be used in the generation of the media stream media samples and making 
the timestamp information associated with each sample. A completely different hardware 
clock, however, will be used to render the media stream. Though the two hardware clocks 
may ostensibly run at the same rate, variations in physical clock rates may impact processing. 
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Rate matching, then, is the concept of compensating for actual frequency variations 
in physical clocks. For example, if a stream of live audio is being received through a 
network connection for rendering on local audio hardware, the rate at which the siream is 
being received may not match the rate at which the local audio hardware actually renders the 
stream. This is because the crystal on the audio hardware has nothing to do with the crystal 
used on the audio hardware on the other end of the network connection. 

In some instances it is possible to perform fine adjustments on the local audio 
hardware in order to attempt to move it closer in frequency to what the source is producing. 
Not all audio hardware will allow this, however, and those that do have certain frequency 
resolution constraints that limit the accuracy of an adjustment and may have limits as to the 
amount of adjustment allowed. 

Processing "drift" occurs due to this rale differential and ways are sought to 
determine the amount of drift accurately and quickly. Further, compensating for the drift 
optimally occurs in frequent minute increments, as opposed to infrequent gross adjustments, 
in order to avoid noticeable perception by an end user. Such perceptible adjustments come 
in the form of "skipping" of sound data and/or "jerkiness" of video images. 

Yet another problem encountered with processing media or data streams is that of 
translating one time frame to another. This often occurs during synchronization as the 
presentation times of a "slave" stream are converted to the time frame of the master stream. 
Ways are sought to achieve such translations with mirdmal error introduction. 

In a multimedia environment, it is advantageous to interconnect software drivers so 
that processing may occur in software drivers that run in a operating system mode with a 
much higher run priority and little security protection to allow access to actual hardware that 
the drivers, in many instances, directly manipulate. Many applications are benefited by 
running in this looser and more performance-oriented mode, generally referred throughout, 
in Windows NT terminology, as "kernel mode." Other robust operating systems will have 
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a functionally equivalent mode. Such interconnected software drivers have the same need 
for synchronizing multiple media streams and for rate matching of physical clocks. 

One prime example of a program currently incapable of easily using kernel mode 
drivers, used throughout this application, comprises graph building functionality that allows 
a user to select and connect together different processing blocks, called filters, to 
successively manipulate a stream of multimedia data. The data typically is a series of 
samples representing sound or video and the processing blocks may include decompression 
processing for compressed data, special effects processing, CODEC functionality, rendering 
the data into analog signals, etc. 

Such filters are typically located in user mode so that the graph builder portion of 
the program may interconnect and control their operation and be responsive to user input and 
rearrangement of processing blocks. Because of the consistent stream nature of multimedia 
data and the generation of large quantities of data, performance is a critical issue. Jn a 
general purpose operating system, system performance caused by repeatedly 
passing/switching back and forth between user mode and kernel mode can be so degraded 
as to prohibit certain multimedia applications. 

Ways to standardize the interconnection of kernel mode filters would be desirable 
and would further benefit by a standardized clock mechanism to assist in solving the 
syncluonization problem associated with processing multiple media streams simultaneously. 
Moreover, such standardized clock mechanism should also address and solve the rate 
matching problem associated with originating the data and rendering the data using different 
physical clocks. 
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SUMMARY AND OBJECTS OF THE INVENTION 

It is an object of the present invention to synchronize processing of multiple data 
streams, each composed of data samples having time interval information associated 
therewith. 

It is another object of the present invention to facilitate matching the processing rate 
of a data stream where the data stream was created by an originating physical clock as 
represented in the stream data samples and a rendering physical clock associated with a piece 
of hardware. 

It is a further object of the present invention to provide a useful clock mechanism 
having a standardized interface for use in a system of interconnected kernel mode software 
drivers. 

It is yet another object of the present invention to facilitate translation from time on 
one clock mechanism to time on another clock mechanism. 

Additional objects and advantages of the invention will be set forth in the 
description which follows, and in part will be obvious from the description, or may be 
learned by the practice of the invention. The objects and advantages of the invention may 
be realized and obtained by means of the instruments and combinations particularly pointed 
out in the appended claims. 

To achieve the foregoing objects, and in accordance with the invention as embodied 
and broadly described herein, a method and computer program product for synchronizing the 
processing of multiple data streams and matching disparate processing rates using a 
standardized clock mechanism is provided. The clock mechanism can be implemented in 
a variety of different ways and is explained throughout in the context of interconnectable 
kernel mode drivers providing efficient data stream processing. 

The clock mechanism of the present invention maintains three different time values 
and makes them available to other components. The time values may be queried or the clock 
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mechanism may send notifications based on arriving at a given time value or at periodic 
intervals. 

The first time value available at the clock mechanism of the present invention is a 
positional time value that is based on the time interval information associated with a data 
stream and reflects the "position" within the data stream being processed The second is a 
physical time value based on an actual hardware oscillator or clock, such as the PC clock or 
a rendering processor clock. Finally, the third is a correlated time value that provides a 
designated time value, such as the positional time value, together with a reference time value 
based on a commonly available hardware clock such as the PC clock. 

With the clock mechanism as a master clock, the positional time value can be used 
by other processing components processing other data streams for synchronization. Since 
the positional time value is based on processing of a data stream, when the processing of the 
underlying data stream stops the clock does as well. Furthermore, timing event notifications 
will also be frozen on an absolute time basis, but will continue to be perfectly timed with 
respect to the positional time value. This provides a great benefit in that "paused" time need 
not be tracked, manipulated, or otherwise be of concern. "Slave" clock mechanisms and 
components will base processing on time as it is provided by the positional time value. 

The physical time value of the underlying hardware clock associated with a data 
stream rendcrer or other processor can be used to match the rate of the data stream being 
processed with the processing rate of the rendering processor. Since the data stream has time 
interval information associated therewith, the originating processor's processing rate is 
inherent therein and may be ascertained and compared to the rendering processors actual 
processing rate as found in the physical time value. This allows any processing rate "drift" 
to be quickly and accurately determined thereby allowing processing adjustments to be made 
with less noticeable effect on the rendered data. 
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Finally, the correlated time value provides a means for other clock mechanisms and 
processing components to translate time information from one basis to another. The 
common hardware clock must be available to other clock mechanisms or components so that 
it may be used as a common reference. Since the correlated time is returned or accessed as 
an atomic operation, any errors introduced are minimal and non-cumulative. 

Two variations of correlated time arc supported in the described embodiment, both 
using the PC clock as a reference. One uses media time as the designated time and the other 
uses the physical time as the designated time. Tluoughout this application, correlated time 
used alone could refer to either of the two variations or even other variations that will be 
apparent to those skilled in the art When distinguishing between the variations, the terras 
correlated media time and correlated physical time will be used with the respective 
designated time compared to the common reference. 

One embodiment of the present invention occurs as part of connecting kernel mode 
drivers in a standardized fashion. A given driver or filter will support and define connection 
point "pin factories" that may be used to instantiate connection pin instances that are 
interconnected to other pin instances on other drivers to allow processing messages to be 
consecutively processed in kernel mode by the drivers without necessary resort to a user 
mode agent. 

A third party agent desiring to connect compliant drivers will query the drivers of 
their capabilities. Such capabilities include what kinds of connection pin instances may be 
instantiated, including their relevant characteristics, such as type of data handled, data 
formats, transfer rates, medium or mode of transfer, input or output nature of a connection 
pin instance, etc. Also queried will be the data buffering requirements and capabilities for 
allocating buffers available at each connection pin factory. 

Once a third party agent, typically ranriing in user mode, has queried the capabilities 
of one or more compliant drivers, the agent will determine the best connection characteristics 



(74) *$ffl¥- 1 0-2 83 1 9 9 

for "chaining" multiple drivers together so that data may be optimally processed between 
them. This determination step occurs after all driver capabilities have been queried so that 
the optimal connection criteria may be selected. 

For each connection pin instance, the third party agent will also determine whether 
a buffer allocator need be created on a particular pin instance. Again, this is done after 
having queried all the different filters and pin fectories prior to making any interconnections. 

The third party agent then interconnects the drivers by creating a connection pin 
instance based on the connection pin factories on each driver. The agent will specify a data 
format and a connection format as part of the connection pin instance creation. In an 
exemplary embodiment implemented under the NT operating system, an actual connection 
pin instance is created by a create I/O operation that returns a handle to a "file." The create 
I/O request will contain the driver instance handle and reference to a data structure indicating 
data formal and connection format for the connection pin instance. 

Furthermore, reference to previously created connection pin instances {e.g. , input 
pins or IRP "sink" pins) will be specified in requests for creating other connection pin 
instances (e.g., output pins or IRP "source" pins) in order to effectuate a connection between 
connection pin instances. After a source pin instance is created using a reference to an input 
pin instance having a buffer allocator, indication is made with the source pin instance of the 
buffer allocator so that data may be transmitted into the new buffer from the existing buffer. 
If no reference is indicated, the source pin instance will leave the data in the existing data 
after processing. 

In order to create a compliant driver, a driver developer wit) support certain standard 
facilities to allow a user mode agent to query capabilities and make interconr*xtions between 
drivers. In one embodiment, built on the Windows NT operating system, this is achieved by 
use of "sets" {i.e., property, method, and event sets) that implement the desired functionality. 



(75) 1#BfJ¥ 1 0 -2 83 1 99 

A set is logically defined as having aGUID (globally unique identifier) to identify 
the set as a whole and a RUID (relatively unique identifier, e.g.. relative within the set itself) 
for each element of functionality within the set. Each set is associated with only one or two 
IOCTLs (lO Controls), and an JOCTL combined with a set specification controls all 
interaction with the driver. 

As currently embodied, three types of sets arc utilized, namely, property sets, 
method sets, and event sets. Property sets arc used for managing values or settings within 
the driver, such as sound volume, transfer rate, etc., and use a single IOCTL with a flag 
indicating whether the call is getting a property value and or setting a property value. 
Method sets are used for managing the operations that a driver may perform, such as 
allocating memory, flushing buffers, etc., and uses a single IOCTL to call the specified 
method. Event sets are used for managing events associated with driver processing, such as 
device change notification, data starvation notification, etc., and uses two IOCTLs, one for 
enabling a specified event and one for disabling a specified event. 

To use a set, an T/O control operation is initiated using the specified IOCTL and 
reference to a data structure having the set GUID, RUID, and other necessary data. For 
example, setting a volume property on a sound card driver would entail an I/O control 
operation using a set property IOCTL, specifying the appropriate GUID for the property set 
having the volume setting, indicating the specific RUID within that set indicates the volume 
property, and containing the new volume setting value. 

To query the sets supported, a null GUID is used along with a query flag on a 
specified IOCTL for a particular set type (e.g. , property set IOCTL, method IOCTL, or event 
enable IOCTL) and a list of set GUIDs supported will be returned. To query supported 
properties, methods, or events within a given set, the set GUID, set type IOCTL, and a query 
flag are used with the operation returning a list of supported RUIDs. 
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By using the generic set mechanism, a minimum of functionality may be 
implemented to support a compliant driver but still allow unlimited extensibility. A set may 
be defined in a written specification that can be independently coded by a multitude of 
different driver developers to create a system of interoperable and interconnectable drivers 
as long as particular sets are implemented. Furthermore, the specification can define 
mandatory properties, methods, and events that must be supported as well as optional 
properties, methods, and events that can be implemented depending on the driver functions 
and advanced capabilities. In addition to the basic minimum commonality required, driver 
developers may incorporate additional functionality by defining their own sets and assigning 
them a GU1D. By being able to enumerate supported functionality (i.e., make queries for 
supported GUlDs and RUIDs ), a caller, such as a third party controlling agent, can adjust 
expectations or make appropriate compensation depending on the capabilities of the 
underlying fillers. 

An embodiment of a clock mechanism having a positional time value, a physical 
time value, a correlated positional time value, and a correlated physical time value may be 
created on a compliant connection pin instance. A "file" is opened on the particular device 
in the I/O indicating the handle of the file object r^resenting the connection pin instance as 
parent and a GUID string value indicating the clock mechanism file type. A handle is 
returned to a file object representing the clock mechanism and the filter will support, through 
this file object, property sets for providing the respective time values. The positional time 
value will be based on the data stream processed by the filter and passing through the 
particular connection pin instance. The physical time will be based on a hardware clock or 
oscillator associated with the filter, if present, or the PC clock. Both of the correlated time 
values will typically use the PC clock as the reference value due to its widespread 
availability. 
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The clock mechanism may be used as a "master" allowing other clock mechanisms 
or components to "slave" off or synchronized with it's time tracking. Furthermore, a default 
implementation based on the PC clock is provided. 

These and other objects and features of the present invention will become more fully 
apparent from the following description and appended claims, or may be learned by the 
practice of the invention as set forth hereinafter. 

In order that the manner in which the above-recited and other advantages and objects 
of the invention are obtained, a more particular description of the invention briefly described 
above will be rendered by reference to specific embodiments thereof which are illustrated 
in the appended drawings. Understanding that these drawings depict only typical 
embodiments of the invention and are not therefore to be considered limiting of its scope, 
the invention will be described and explained with additional specificity and detail through 
the use of the accompanying drawings in which: 
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detailed nKscRrmoN of the ereeebred em&qbimei£I S 

As used herein, the term "user mode" refers to a level of operation in an operating 
system where most user written programs rua The user mode level of operation is typically 
the most secure level and has a significant amount of overhead to prevent one application 
program or process from interfering with another application program or process. 
Furthermore, access to system resources is highly controlled through specific interfaces and 
run priority is generally one of the lowest, if not the lowest. 

As used herein, the term **kernel mode'* refers to a level of operation in an operating 
system having significantly less restrictions than the user mode level of operation. Examples 
of kernel mode programs or processes would include software drivers for controlling 
hardware components. Typically, kernel mode program? are performance sensitive, and 
therefore, have less operational overhead than user mode programs. Furthermore, access to 
hardware and many system resources is unrestricted or much less restricted than for user 
mode programs. In many instances, program code running in kernel mode relies on 
programmer discipline and conformity to convention in order to establish good system 
behavior (e.g., not disrupting another program's address space, etc.). Another term used for 
kernel mode is "trusted" code. 

As used herein the term "driver" refers to software driver programs typically running 
in kernel mode. The teem driver may also refer to the actual executable program that is 
loaded onto the operating system or a portion thereof that imparts certain functionality. 
Drivers are in many instances, though not necessarily, associated with some form of 
hardware. 

As used herein, the term "filter 51 refers to a portion of the functionality found within 
a software driver, including the entire driver itself, where connection points may be exposed 
for sending data through the filter. For example, a software driver may support a number of 
different filters or may have one single function. Furthermore, a number of filters from 
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different drivers that are internally connected together and externally exposing connection 
points for input and output may collectively be referred to as a single filter. Also, in a more 
generic sense, the term filter may refer to the operation performed, such as decompression, 
etc., regardless of whether that occurs in a software driver filter running in kernel mode or 
another piece of program code running in user mode. 

As used herein, the term "driver object" refers to an operating system entity, defined 
by the operating system, for managing and making known a software driver as a system 
resource. 

As used herein, the term "device object" refers to a system level entity defined by 
the system, for making known a portion of a drivers functionality available for use as a 
system resource and defines the driver functionality and availability to other system 
components. Both the driver objects and device objects are typically created upon loading 
and initialization of the driver software. 

As used herein, the term ll file object" refers to an operating system entity, defined 
by the system, for managing an invocation of a resource specified by a device object. A file 
object provides a context for usage of a driver object. Furthermore, a file object may be 
hierarchically related to another file object if the previous file object is designated as a 
''parent" during the creation of the new file object. File objects are typically used in 
managing all I/O operations that operate on a stream of data. 

As used herein, the term "data" refers to any information that is processed through 
the interconnected kernel mode filters. Such data includes media data representing video, 
audio, text, MIDI, etc. but may also include control information or parameters for other 
applications. For example, a kernel mode filter graph may be used in process control 
operations where the control information passed between the different filters is used to 
develop control signals for actuating machinery. While examples are given of media 
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processing systems, other applications could in Jikc manner benefit from the system of 
interconnected kemel mode filters explained herein. 

Throughout this application, the description of the present invention is described 
within the context of the Windows NT™ operating system available from Microsoft*, 
Furthermore, familiarity with the Windows NT I/O architecture is presumed in order to 
understand the preferred embodiment explained herein. A good tutorial of the I/O system 
as well as the NT operating system in general can be found in the book "Inside Windows 
NT' written by Helen Custer and published by Microsoft Press and is further incorporated 
herein by reference. 

While the discussion of the drivers and system entities such as file objects, device 
objects and driver objects are explained herein within the context of how they operate in the 
Windows NT operating system, those skilled in the art will appreciate that the present 
invention may be implemented on other operating systems having analogous components, 
whether or not they use the same terminology. For example, should another operating 
system have an entity that operates as a file object, it could be interpreted as a file object 
regardless of its actual title. 

Referring now to Figure 1, an example system is presented for reading a stream of 
sound data from a disk drive and rendering that sound data so that it may be heard through 
a speaker according to the prior art model. An amount of data is stored on hard drive 20 
representing sound in the form of digitized sound samples. Alternatively, the source of the 
sound data stream may be digitized information coming over a phone line, digitized 
information from network or other communication packets, or other sources known in the 
art. The data stream is composed of digitized samples having time interval information 
associated therewith either by data format and convention or by explicit timestamp 
information attached to each sample. A kernel mode disk driver 22 interacts with the disk 
drive hardware 20 and is under control of a user mode reader program component 24. A 
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controlling agent 26 manages the different componems in order to effectuate the rendering 
ofthe sound data and may include dynamic graph building capabilities so that the different 
software components may be dynamically allocated in order to provide custom filtering or 
other processing paths as designated by an end user. 

The reader component 24 wiJl interact with disk driver 22 using a standard I/O 
control interface ofthe operating system and will cause the compressed sound data to be read 
from the disk drive 20 into buffers allocated in user mode as part ofthe user mode process 
address space. Next, a decompressor component 28 will decompress the compressed data 
into its decompressed format for processing. As shown, this step is done entirely in user 
mode with the attendant lower priority and process behavior safety mechanisms. 

The effects filter 30 will operate on the data to provide some special effect and will 
have an accompanying effects filter 32 operating in kernel mode. Furthermore, an effects 
processor 34 may be present or the effects filter may operate entirely in software emulating 
the actual hardware processor. In order to access the effects filter 32 the effects component 
30 will use the system 170 control mechanism to transfer the data and control to the effects 
filter. Again, the kernel mode/user mode boundary is crossed in order to make this 
transition. 

The effects filter 32 will control the effects processor 34 and cause whatever special 
effect is necessary or desired to be made on the data This may entail copying all the data 
from the effects component 30 down to the effects filter 32 and again to the effects 
processor 34 depending on actual system configuration. While many software effects 
components have a hardware processor associated therewith, others function entirely within 
system software running on the host processor. 

After control and the data is transferred back into user mode at the completion ofthe 
processing of the effects component 30, it is then transferred to sound rendering 
component 36. The sound rerklering component 36 will transfer the control and data to the 
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sound rendering driver 38 which in turn controls the sound card 40 in order to render the 
data, as processed and filtered, as sound through speaker 42. As can be readily seen, there 
exists a variety of transfers between user mode and kernel mode that add inefficiencies to the 
rendering of the sound data. Because of the liming sensitive nature of multimedia data, such 
as a continuous stream of sound or video, it is advantageous to reduce these inefficiencies 
and transitions of control as well as the multiple copying of data between different buffers. 

One embodiment of the present invention and used throughout will consist of a 
service provided on the Windows NT operating system architecture. This service is broken 
into different software components that a user ofthe system will access. First, a user mode 
API is available that will include a routine for creating connection pin instances and other 
file objects representing particular ftinctionality, such as a clock mechanism or a buffer 
allocation mechanism. Additionally, and more importantly, there will be a complete set of 
routines and data structures to assist the driver developer in making drivers that are 
compliant with the standardized architecture. By utilizing such facilities from the system, 
different driver developers may create compliant drivers that will interact with one another 
according to the specified architecture. User mode agents communicate with compliant 
drivers through an environment subsystem running in user mode that will communicate with 
the system services ofthe NT executive and the I/O manager. This is the same standard I/O 
mechanism for all other I/O and the present implementation ofthe preferred embodiment 
will utilize existing system services as much as possible. 

The architecture of the system of Figure 1 utilizing the present invention will appear 
as shown in Figure 2. A controlling agent 44 will query the drivers known in order to then 
make interconnections according to data format and connection format to effectuate the 
rendering entirely in kernel mode. Furthermore, the controlling agent will receive 
notifications of important events so that it may exercise control as necessary. Examples of 
such events would include end of processing, a data starvation situation, etc. 
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In this configuration, the sound data is read from disk drive 46 by the disk driver 48 T 
as before. A reader driver 50 controls disk driver 48 and is "vertically" associated with disk 
driver 48 according to the NT layered I/O architecture as used in conventional fashion. The 
terms vertically and horizontally are used to distinguish driver connections that currently 
occur as part of the NT layered I/O architecture (vertical) and connections according to the 
interconnected kernel mode drivers made dynamically by a third party controlling agent 
(horizontal). 

Reader driver 50 is also interconnected "horizontally" to a decompressor driver 52 
according to the connection methods explained hereafter and is managed by the controlling 
agent 44. Decompressor 52 will perform the decompression in kernel mode before passing 
the data and control to the effects filter 54. The effects filter will apply the special effects 
utilizing an effects processor 56 as necessary before passing the data and control to the sound 
rendering driver 58 that controls the sound card and causes the data to be rendered as sound 
on speaker 62. As can be noted by reference to Figure 2, keeping processing in kernel mode 
represents an efficiency advantage by eliminating multiple transitions between user mode 
and kernel mode and by reducing ihe amount of overhead normally associated with 
processing in user mode. 

Referring now to Figure 3, a logical diagram showing the hierarchal nature of 
system objects related to interconnected software drivers compliant with one embodiment 
of the present invention is shown. A driver object 64 is created to represent the executable 
software code image as loaded in memory. The driver code image contains the entirety of 
the driver's functionality, and the driver object 64 includes information regarding the image, 
such as its location on the system, the kinds of devices supported, etc. 

For each type of independently accessible functionality by a controlling agent, 
device objects 66a-66N are created in the I/O directory structure representing the different 
functions that are available that will be accessed by user mode clients. These typically 
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represent filters or other portions of functionality independently available. The driver object 
64 and the device objects 66a-66N arc created upon installation of the driver code as 
represented by the enclosing box 68. 

Historically, a device object exists for each element of physical hardware. The 
flexibility in modern I/O systems, however, allows a device object to represent a filter 
implemented entirely in software. As such, device objects may be readily created for each 
instance of a filter implemented solely in software. A software filter may therefore be 
implemented so that each instance as represented by a device object has a one-to-one 
correspondence with a device object or a single device object may follow the more 
traditional approach and manage multiple file objects, with each file object representing a 
client instance of the filter. 

Upon a device object, as shown for device object 66a, file objects are created 
representing independent instances of the functionality represented by device object. While 
a device object represents a filter and may manage multiple instances of that filter, a file 
object represents the actual instance of that filter used by a particular entity. Therefore, file 
object 70 is an instance of the filter defined by device object 66a. 

To use a filter, the controlling agent or other user mode client opens a file on a 
device available in the I/O directory structure. A file object with appropriate context 
information will be created and a handle to that file object returned to the user mode client. 
While file objects may be hierarchaHy related by specifying a "parent" File object during 
creation, file objects will also have a sibling relationship in that they are all children of the 
same device object 

Context information within a file object consists of iriformation to manage the I/O 
interface with user mode clients, the "state" of the entity that the file object represents, etc. 
The context information has system required information and further includes user definable 
areas that can be given special meaning. An example of how the user definable area can be 
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used will be shown hereafter discussing the implementation of a validation and IRP routing 
method. 

In order to provide connection pin instances, the file object 70 representing a filter 
instance will be used as a parent in creating children file objects representing the connection 
pin instances for a particular filter. While file object 70 will be queried for the connection 
pin factory definitions and availability, actual file objects will be created for each instance 
of such a pin factory, using the particular file object as the appropriate informational context 
in order to validly and correctly create the connection pin instance. For example, file objects 
72 and 74 represent connection pin instances for the filter represented by file object 70 and 
are hierarchaJry related to file object 70. The correction pin instances, as represented by file 
object 72 and 74, respectively, may be a data path into and then out of the filter instance 
(represented by file object 70) which can be used for connecting to other connection pin 
instances in forming a series of chained filters or other driver functionality. 

Just as a pin instance is represented by a file object having a rnerarchial relationship 
to another file object representing the filter instance in order to provide context information 
for the pin instance, other file objects may be hierarchically related to a pin instance in order 
to represent other functionality so that proper context information is available. Context 
irrformation is necessary to distinguish one pin instance from another according to the 
individual parameters used in creation, such as pin data format, communication type, etc. 
Other operational entities, such as a buffer allocation mechanism, a timing 
mechanism, etc., requiring either an individual context or user mode control through a handle 
may also be represented by file objects. Furthermore, hierarchical relationships between the 
file objects (e.g., a buffer allocation mechanism associated with a particular connection pin 
instance) may be established if necessary by specifying a parent file object during creation 
of the child file object. These parent/child relationships exist to determine relationship and 
structure betvveen the file objects representing the orxrational entities. Additionally, a 
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particular type of "parent" file object will only be able to produce certain types of "children" 
file objects, thus requiring the creation validation mechanisms as explained hereafter. Again, 
such file objects have corresponding handles available in user mode that are returned to a 
client through a system API call such as NtCreateFile. 

The handles to file objects are used by user mode clients, such as a controlling agent, 
to communicate with the kernel mode drivers. The hierarchical chain of file objects, device 
objects, and driver objects allows the I/O subsystem to traverse back to the driver object 
through the hierarchically related file objects and device objects to arrive at the entry points 
into the actual driver code. Such entry points are references (e.g., pointers) to functions in 
the software driver code. Furthermore, each of the objects in the object pathway between a 
particular file object and the driver object having the entry points to the software driver code 
provides important context information for the I/O subsystem in creating IRPs as well 
references into data structures used for properly routing IRPs according to the routing and 
validation mechanism explained hereafter. 

Handles for file objects and other system objects arc process-specific and are the 
means by which a user mode process will communicate with an underlying object. It is 
interesting to note that multiple handles may be created to reference a single underlying 
system object, such as a file object This means that multiple applications may be feeding 
information to a pin instance as represented by a file object. 

One element of information that is important for interconnecting different drivers 
is the device object stack depth parameter. This will indicate the IRP stack location of a 
particular driver object In this manner, a single IRP may be used and passed between 
interconnected drivers using the I/O manager, thereby providing a performance enhancement 
over separately creating and sending CRJPs between the various interconnected drivers. 
Alternatively, each driver could create through appropriate I/O manager calls new IRPs for 
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each successive communication and cause each new IRP to be sent to the next driver in the 
chain of interconnected drivers. 

Referring now to Figures 4A-4C, extensions to the system driver objects, device 
objects, and file objects are shown that allow validation of file object creation of differing 
types as well as I/O Request Packet (IRP) routing to appropriate handlers. Figure 4A shows 
a driver object 76 representing the executable code implementing one or more filters or other 
driver functionality. Within the driver object, the Windows NT architecture requires a 
reference to a create handler provided by the software driver developer. According to this 
embodiment, a multiplexing dispatch function 78 is referenced from the driver object 76 as 
the create handler and will be used to route messages to particular create handlers depending 
on the file object type to be created. Operation of the multiplexing dispatch function 78 will 
be explained in connection with the flow chart shown in Figure 6 hereinafter. 

In like manner, other handlers from the driver object will indicate a multiplexing 
dispatch function and, depending on implementation, they may be the same function. Tn 
other words, as explained in more detail below, each type of VO handler reference (e.g. , read, 
write, device control, etc.) will point to a multiplexing dispatch function that uses the 
extension data in a device object and the context information in a fife object in order to route 
the message to the appropriate handler. The extension data in the device object that 
references a validation table will be used when no parent file object is specified on a create 
operation. Otherwise, the parent file object context information will indicate the correct 
validation table. 

Figure 4B shows a driver object 80 which has a particular device extension area 82 
that can be utilized as desired by the driver developer and includes driver specific 
information. At a defined location within the device extension area 82 of the driver object 80 
is a reference to a data structure, known as a file type validation table 84, containing string 
representations of file object types 86 and references to the associated create handlers 88 for 
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each Hie type represented. The create multiplexing dispatch function will the utilize file type 
validation table 84 to validate the Hie object type to be created and then turn control over to 
the appropriate create handler as will be explained in detail hereafter in connection with the 
discussion of Figure 6. The string to be validated is found in the IRP create request and 
originates from the file name string used with the NtCreateFile function call in user mode. 
The NtCreateFile call is made within the user mode function cell to create a pin instance or 
other mechanism. 

Figure 4C shows a file object 90 having a file context area 92 that is free to be used 
by the software driver developer. Reference is made from the file context area 92 to an IRP 
request handier table 94. The different types of IRP request 96 are associated with references 
to particular handlers 98, and the appropriate multiplexing dispatch function will use this 
information to access the correct handler. In the case of determining the correct create 
handler, a data structure known as a file type validation table 100 is referenced having string 
representations of file object types 102 and references 104 to the associated create handlers 
for each file type represented. For children file objects (i.e., file objects that have another 
file object rather than a device object as parent), the type is represented by a suing that is 
compared to the strings in the file object types 1 02. When a match is found, the associated 
create handler is accessed using a reference from the references 1 04 that is associated with 
the matched file object type string. If no match is found, then the request is invalid and an 
error indication made. 

Referring now to Figure 5, the installation procedure for setting up the creation 
validation and mechanism is shown. At step 1 06, the installing program will make reference 
in the driver object to the appropriate multiplexing dispatch functions. As shown in Figure 
4A, the create handler points to a generic multiplexing dispatch function. In like manner, all 
other handler references in the driver object 76 would point to other generic multiplexing 
dispatch functions germane to the particular handler as necessary. Alternatively, each 
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handler reference could point to the same multiplexing dispatch function that could in turn 
process the IRP request and route it to the appropriate handler. Such an alternative 
multiplexing function will necessarily be more complex in order to accommodate different 
kinds of request (e.g., create, write, etc.). 

Next, at step 1 08, each device object created as part of the software driver executable 
code installation will be adjusted to reference the file type validation table 84 as shown in 
Figure 4B. Finally, at step 110, the processing of IRP requests will begin with the 
multiplexing dispatch function using the file type validation table 84 as referenced from the 
appropriate device object 80 

When a file object is created, the appropriate IRP dispatch table 94 will be created 
and referenced along with the indexed file object type validation table 100 as necessary. The 
creation of the file object type validation tables occurs within the provided create handlers 
according to file object type. The data structures are created representing the IRP dispatch 
table 94 and the file object type validation table 100 and a reference thereto stored at a 
specific location with the file context information 92 of the particular file object 90 being 
created. 

Referring now to Figure 6, a flow chart is presented showing the operation of the 
create multiplexing dispatch function and its validation mechanism including its interaction 
with the data structures referenced from the system driver objects, device objects, and file 
objects. At step 112, a user mode process sends an I/O request for creating a file object. 
This I/O create request is made using an invocation to the system API for NtCreatcFilc. At 
step 114, the I/O manager sends the IRP to the multiplexing dispatch function 78 based on 
the reference in the driver object 76 (see Figure 4A). 

Once the multiplexing dispatch function 78 has the IRP for the create request, a test 
is made at step 1 1 6 to determine if there is a parent file object. The information necessary 
to make this detennination will be found within the IRP itself and originally be supplied by 
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the user mode process. The user mode process will supply a handle referencing the "parent" 
file object as part of the create request and the NT system will create the IRP having the 
correct reference to the "parent" file object. 

If there is no parent file object, the right branch is taken, and the multiplexing 
dispatch function 78 uses the device extension S2 from the appropriate device object 80 to 
reference a file type validation table 84 (see Figure 4B) at step U 8. Using the validation 
table 84, the multiplexing dispatch function 78 will validate the file object type at step 120 
by comparing the string in the request with the file object types 86 strings. 

If there is a matching string as determined at step 1 22, the appropriate create handler 
is accessed at step 124. Otherwise the create request is rejected at step 12$. The create 
handler as accessed at step 124 will create, or cause to be created, the file object at step 126. 
With the created file object, the appropriate create handle will make the file object reference 
in the file context 92 to an IRP dispatch table 94 that it has previously created. 

Again at step 1 16, it may be determined that there is a parent file object present. If 
a parent file object is present, as determined at step 11 6 as found in the IRP associated with 
the create request the multiplexing dispatch function 78 uses the file context 92 ftom the 
parent file object 90 to reference an JRP dispatch table 94 {see Figure 4C) at step 1 30. For 
a create request, the multiplexing dispatch function 78 will access a file type validation table 
100, at step 13Z Using the file type validation table 100, the multiplexing dispatch function 
78 will validate the file object type at step 133 by comparing the string in the request with 
the file object types 1 02 strings, as was done above. 

If mere is a matching string as determined at step 1 34, the appropriate create handler 
is accessed at step 138. Otherwise the create request is rejected at step 136. With the 
appropriate create handler, the file object is created at 140, and the create handler will make 
a new IRP dispatch table 94 for the newly created file object and will make reference in the 
newly created file object 90 file context area 92 to the newly created IRP dispatch table 94 
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m step 142. Note that the same fiJc object structure as shown in Figure 4C is used to explain 
interaction with both the parent file object and the validly created child file object. While 
the same structure exists in hoth cases (once the new file object is created), they will be used 
differently and contain different irrfbrmation. 

Whenever a connection pin instance is crcated > a connection pin ID is passed that 
identifies the pin factory in the filter that "supports" the creation of the pin instance. Those 
skilled in the art will note that the connection pin ID may also be validated as a string in a 
validation table in much the same manner as the file object is validated and that other 
implementation variations exist. 

In order to make corrections between different drivers, a common mechanism must 
be present to assure that a given driver supports such intercoimections. This common 
mechanism must allow discovery of filter capabilities including connection pin factory 
capabilities. Furthermore* such a mechanism should also be extensible to provide additional 
flexibility to driver developers. 

One mechanism chosen in the present embodiment for defining compliant drivers 
and allowing discovery of capabilities are identified "sets" of related items. This is a 
convenient mechanism to be used with existing I/O communication mechanisms. A set is 
logically defined as having a GUID (globally unique identifier) to identify the set as a whole 
and a RUID (relatively unique identifier, eg., relative within the set itself) for each element 
of functionality within the set The set identifier and any other data structures necessary for 
operating with the chosen RUID item are passed as part of an I/O control call using the filter 
handle as a parameter. Only a small number of IOCTU need to be allocated in order to 
implement a full system of functionality. As implemented, three different types of sets are 
established depending on their functions, requiring a total of four IOCTLs. Other 
implementations may use sets in a different manner. The particular IOCTL will signal the 
handler for I/O control to interpret or use the chosen element (using the RUID) in a certain 
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manner. Furthermore, control flags may be passed with the GUID and RUJD to norther 
specify control information. 

The first set type is a property set and is used in connection with values or settings 
found within the driver or on any associated hardware. Examples of such settings would be 
transfer rate, volume level, etc. One IOCTL is associated with property sets with a control 
flag differentiating between a "get" property and a "set" property command. In this manner 
the same data structure can be used to either set or get a particular property with the driver 
determining the action required based on the IOCTL used. The correct property is identified 
by the set identifier consisting of its unique GUID and RUID combination. 

Method sets are another type of set used and are a set of actions that can be 
performed by a driver. Only one IOCTL is needed to identi jy the method set with the correct 
method to be actuated identified by the unique GUID and RUID combination for the set 
identifier. Methods arc used to control the driver and include such functions as initializing 
the driver for use, clearing buffers, etc 

Event sets arc used for managing events associated with driver processing, such as 
device change ratification, data starvation notification, etc., or any other notification defined 
by the set that may be useful to a user mode application. Two IOCTLs are used, one for 
enabling a specified event and one for disabling a specified event, while any data structures 
necessary for a given event identified by a RUID can be shared whether enabling or disabling 
the event. 

To use a set, an I/O control operation is initiated using the specified IOCTL and 
reference to a data structure having the set GUID, RUID, and other necessary data (e.g„ 
control flags, data structures, etc.). For example, setting a volume property on a sound card 
driver would entail an I/O control operation using a property set IOCTL, a control flag 
indicating a set property operation, the appropriate GUID for the property set having the 
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volume sening, the specific RUID within that set indicates the volume property, and the new 
volume setting value. 

To query the sets supported, by type, an IOCTL for a particular set type (e.g., 
property IOCTL, method IOCTL, or event enable IOCTL) having a null GUID and control 
flags to indicate supported set enumeration are issued as part of an I/O command and a list 
of set GUlDs supported will be returned. To query supported properties, methods, or events 
within a given set the set GUID ; set type IOCTL, a null RUID, and control flags indicating 
enumeration of supported elements are used with the I/O operation. A list of supported 
RUlDs will be rerurned as a result of the I/O operation. This will allow a third party agent 
to determine which, if any, optional elements of an implemented set are supported. 

The written specification of a set uniquely identified by a GUrD allows a 
documented mechanism that both driver developers and third party controlling agents may 
use as an implementation guide. The third party developer will know of a given driver's 
capabilities based on response to queries and preprogrammed knowledge based on the 
abstract set definition. Likewise, a driver developer may use the abstract set definition as 
a guide to implementing a set or group of sets providing known functionality to any third 
party agent. 

In order to provide the connection abilities described herein, a compliant driver must 
support certain sets. The following tables illustrate some important kinds of information that 
may be supported in property set format and that can be used in implementing the present 
invention. The first table refers to properties about a connection pin factory that would be 
implemented by a filter, while the second table refers to properties about an actual 
connection pin instance that would be created by using a particular connection pin factory 
as a template. 
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L Filter Properties and Their Use 


Property 


Description 


Connection Pin 
Factories 


Lists the different types of connection pin instances that may be 
created on a particular filter, each distinguishable type referred to as 
a pin factory. Note thai this is not the total number of connection 
pin instances which could be instantiated on this device, but the 
number of unique connection pin types, such as an audio input and 
audio output. 


Connection 
Instances 


Lists the number of instances already created of a given connection 
pm factory as well as the highest number of instances supported for 
that particular connection pin factory. If the total cannot be 
determined until the filter is actually connected, this property will 
return a -1. 


Data Flow 


Lists the possible data flow direction of a connection pin factory 
with respect to a fdter (e.g., into the filter, out of the filter, or either 
into or out of the filter). 
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I Communication 


Lists the communication requirements for a given connection pin 
factory in terms of processing IRPs. Some connection pin factories 
may not be interconnected but have other forms of control 
mechanisms associated therewith such as e "bridge" to a file source 
for data that represents a source point on a graph. The bridge 
control mechanism would allow setting of a filename indirectly 
where information is stored. 

In an exemplary implementation, an agent (which decides which pin 
factory to use for making a connection pin instance) must be able to 
understand the intrinsic meaning of a "none", "sink" or input, 
"source" or output, "both," and "bridge" communication types for a 
connection pin factory. For example, a source connection pin 
instance requires a handle or reference to a sink connection pin 
instance, etc. 

In the communication type context, sink and source refer to the 
disposition of the connection pin instance in processing IRPs. A 
sink would receive the IRPs for processing, while a source would 
pass the IRPs onto the next appropriate processing component. 

There are two communication types that are neither sink nor source 
and represent end points in the connection graph. An end point 
represents the place where data either enters or exits from the 
connected filters, A none designation indicates that the connection 
type may not be instantiated while a bridge communications type 
refers to an end point that may be instantiated so that specific 
properties may be manipulated. For example, a bridge end point 
that is part of a file reader will likely have a property that will 
contain the path and file name oFa file that stores the data to be 
processed. 


Data Ranges 


Lists the possible data ranges that a connection pin factory may 
support, including the format of the data, if relevant. In one 
implementation, a count followed by an array of data ranges, which 
the connection pin type may support, is used as part of the property. 
In that implementation, if different data ranges are supported under 
different mediums or interfaces (see below), different connection 
pin factories are available on a particular filter to accommodate such 
differences. Furthermore, each data range structure may be 
extended for format specific detail such as number of bits and 
channels. 

The actual data format a connection pin instance uses is set during 
creation of the instance. The data range property is used to assist in 
determining what that actual data format should be for a particular 
connection pin instance and is accessed or queried by a third party 
controlling agent 



«f§8¥ 10-283199 



Interfaces 


Lists other set GUlDs indicating the supported interfaces on a 
particular connection pin factory. An interface is the type or types 
of data that may be communicated through a connection pin factory. 
tot example, MIDI data, CD music, MPEG video, etc., would be 
iiu^naccs in me sense uiat data has a particular convention and 
tormat that a filter could handle. Such interfaces also comprise 
protocols for submitting the data. An interface is independent of the 
medium by which it is communicated. 


Mediums 


Lists the supported mediums on a particular connection pin factory 
A medium is the way or mechanism by which information is 
transferred, such as TCP-based, sockets, etc. An interface may be 
defined on top of a variety of different mediums. In the preferred 
embodiment and implementation explained herein, an IRP-based 
medium and file 10- based medium is used. 


Data 

Intersection 


Returns the first acceptable or "best" data format produced by a 
connection pin factory given a list of data ranges. This approach is 
used to allow a third party agent to determine data requirements 
when chaining different filters together. In one implementation the 
data intersection property is used to determine the best data format 
produced by aconnection pin factory given the constraint of a list of 
data ranges. The list of data ranges may be acquired using the data 
ranges property on another pin factory that will be connected as 
explained previously. 

A third party controlling agent, which has no knowledge of the data 
type specifics, may use the data range list of one connection pin 
factory and return the "best" (e.*. , first acceptable data format) data 
format on the current connection pin factory. Although a set of 
ranges of the two intersecting connection pin factories could be 
returned, only the best format is returned by the driver. In this 
manner, the third party controlling agent can apply this "best" data 
format to the next driver in the graph in order to create a virtual set 1 
of connections before actually initiating the creation of connection 
pin instances and connecting the entire graph of filters together 
This allows the controlling agent to assess the viability of a 
particular filter graph selected by a user and point out potential 
problems to the user before actually connecting the graph. The data 
format returned can also be restricted by the formats available given 
the connections already made on the filter. 

This property is capable of returning an error if a particular data 
format cannot be determined until an actual connection is made or if 
an intersection is dependent on multiple data formats on different 
connection points. Essentially, intersection information is provided 
while the property itself will return a data format 
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TAB U 2 



Property 



State 



Connection Pin Instance Properties and Their Use 



Priority 



Description 



Describes the current state of the connection pin instance. Possible 
states include being stopped, acquiring data, processing data, being 
paused or idle, etc. The state represents the current mode of the 
connection pin instance, and determines the current capabilities and 
resource usage of the driver. 

The stop state is the initial state of the connection pin instance and 
represents the mode of least resource usage. The stop state also 
represents a point wherein there will be the most latency in data 
processing in order to arrive at the run state. The acquire state 
represents the mode at which resources are acquired (such as buffer 
allocators) though no data may be transferred in this state. The 
pause state represents the mode of most resource usage arid a 
correspondingly low processing latency to arrive at a run state. Data 
may be transferred or "prerolled" in this state, though this is not 
acttia ly a run state. The run state represents a mode where data is 
actually consumed or produced (U. t transferred and processed) at a 
connection pin instance. 

More resolution in control may be accomplished using custom 
properties depending on the purpose of the filter and tie underlying 
hardware. For example, in order to make an external laser disk 
player spin up, one would set some sort of custom "mode" property 
specific to that class. Setting this property may also change the state 
ot the device but not necessarily, depending on the effect of the 
mode. 



Describes the priority of the connection pin instance for receiving 
access to resources managed by the filter and is used by the filter in 
r^ur<^ocanon arbitration. This property, if supported, allows a 
third party controlling agent to indicate the priority placement of the 
partciilar pm instance relative to all other connection pin instances 
of all other drivers which may share limited resources with this 
particular connection and instance. 

This priority property may also be implemented to allow an agent to 
set ttner tuning of the priority within a single class of priority 7 For 
example, a priority may have certain subclasses associated 
therewith. If two drivers competing for the same resources have the 
same priority class, then the subclass priority is used to determine 
resource allocation between the two drivers. If the subclass priority 
is also the same, then arbitrarily, the First connection pin instance 
will receive the resources. 



Data Format 



Used to get or set the data format for the connection pin instance. 
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The previous tables arc given by way of example and not by limitation, and those 
skilled in the art will appreciate that many different properties and schemes may be 
implemented in order to create the connections between different drivers. One important 
element is the standardization factor so that different driver manufacturers or development 
groups may create drivers that may be interconnected since they are able to implement the 
same property sets. 

Another useful property set gives topology information for the internal relationships 
of input and output connection pin factories on a given filter. This irifbrmation will state the 
relationship of input pin factories and corresponding output pin factories on a given filter as 
well as what type of processing happens between the input and output pin factories. 
Examples of the processing that occurs would be different data transformations, data 
decompression, echo cancellation, etc. Such information is useful to an automated filter 
graph builder that will trace a hypothetical connection path using multiple filters before 
making actual connection pin instances and connections. Essentially, the topology 
information explains the internal structure of the filter and exposes this through a property 
set mechanism to inquiries from third party agents. 

Therefore, a compliant driver is simply one that implements the designated property 
set. This allows a third party controlling agent to make queries and settings to the compliant 
filter once it is determined that a given property set is supported. The overall goal Is to 
acquire enough information on how to connect the differing filters together in order to form 
a filter graph. 

By using the generic set mechanism, a minimum of functionality may be 
implemented to support a compliant driver but still allow iinlimited extensibility. A set may 
be defined in a written specification that can be independently coded by a multitude of 
different driver developers to create a system of interoperable and interconnectable drivers 
as long as particular sets are implemented. Furthermore, the specification can define 



«fH¥ 1 0-2 83 1 9 9 



mandatory properties, methods, and events thai must he supported as well as optional 
properties, methods, and events that can be implemented depending on the driver functions 
and advanced capabilities. Besides the basic minimum commonality required, driver 
developers may incorporate additional Rinctionalily by defining their own sets and assigning 
them a OUID. 

Referring now to Figures 7 and 8, an illustration of the process for connecting two 
kernel mode filters is illustrated. Figure 7 shows a logical block description wherein each 
filter instance and connection pin instance is represented by file objects. Figure 8 is a flow 
chart illustrating the steps to creating the file objects and the appropriate connections. 

Beginning at step 144, an instance of Filter A 146 and an instance of Filter B 148 
are created by a user mode agent. These are created using standard file system APt for 
creating files with a particular device. Filter A 146 and Filter B 148 will be compliant filters 
or drivers because of their implementing the appropriate property, method, and event sets to 
support the creation of connection pin instances and for querying the respective filter s 
capabilities in terms of sets supported and connection pin factories defined for that filter. 

The third party controlling agent will then query Filter A 146 and Filter B 148, 
respectively, at step 1 50 to determine connection pin factories available and the attributes for 
connection pin instances that may be created therefrom. These attributes include, as 
mentioned previously, the connection format and the data format for each individual type of 
pin instance for each respective filter 146 and 148. The querying will be accomplished using 
the set based query mechanisms explained in detail previously. 

After querying such information, the third party controlling agent wiU determine the 
optimal connection format based on the ranges of data formats and connection formats 
previously queried This ^termination occurs at step 1 52 and places in the third party agent 
the ability to use the same filters in different ways according to the needs of a selected 
connection path. The third party controlling agent will use the data intersection property, 
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topology information, and connection pin factories on both the filters in order to determine 
how best to select data format and connection arrangements depending on the actual filter 
graph being made. 

Input filter pin instance 1 54 is created by the third party agent at step 1 56 using the 
optimal detection formation determined at step 152. Since input pin instance 154 is a file 
object, a handle will be returned from the create process that can he used for delivering I/O 
requests to the input instance 154. Furthermore, the creation of the input pin instance 154 
was validated and uses the routing and validity mechanisms sliown previously in discussion 
with Figures 4A-4C, 5, and 6. 

In order to finalize the connection, output pin instance 158 is created at step 160 
using as a parameter in the MCreatcFile call the handle of the previously created input pin 
instance 1 54. The effect of thus creating the output pin instance 1 58 is to utilize the system 
file management and I/O management facilities to create an internal ]RP stack structure that 
allows an original write command to be corjsecutively processed by the variously connected 
connection pin instances and filters in an appropriate order so as to facilitate direct data flow 
between the differing filters. This requires that the input pin instance be created prior to the 
associated output pin instance that will be feeding the input pin instance. 

The stack depth parameter of a device object controls how many stack locations are 
created from IRP sent to this driver. A stack depth parameter is assumed to be one when a 
device object is initially created and may be modified thereafter depending on the whether 
multiple drivers are chained together. In the current system, modification occurs, if 
necessary, when an output pin instance transitions from the initial "stop" state lo the 
"acquire" or other state. Connection pin instance state transition is the mechanism that 
determines correct stack depth parameter information for proper IRP creation and treatment. 

In order to correctly allocate the internal IRP stack structure for a chained set of 
c<mnection pin instances, h is necessary to transition the connection pin instances out of the 
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stop state in a specified order; beginning with the last input pin instance (in this case input 
pin instance 154) and working consecutively backwards to an associated (e.g., connected) 
output pin instance (in this case output pin instance 158). If many filters ore chained 
together, the deepest filter's or bridges input pin instance must be the beginning point of 
transitioning and building successively backwards until the initial output pin instance on a 
bridge or filter is set. In other words, the transition out of the stop state must occur 
backwards up the chain so that each connection pin instance gets the stack size needed after 
the previous connection pin instance. Typically, though not necessarily, a connection pin 
instance transitions from the stop state to the acquire state and for discussion purposes 
hereinafter tnmsitkining to the acquire state will accomplish the same purpose with respect 
to stack depth parameter adjustment as transitioning out of the stop state. 

Once all pin instances are in the acquire state, stream reads and writes may be issued 
to the filter graph. It is interesting to note that the system explained herein allows connection 
of associated input and output pin instances to occur in any order; only the transition from 
the stop state must occur in bottom up or deepest first fashion. Furthermore, the filter graph 
is rcconfigurable to allow changes to be made after initial creation. When changes are made, 
state transitions need only occur on those connection pin instances that are in the stop state 
in order to assure correct stack depth parameter information. 

Connection pin factories found on filters represent places where a filter can consume 
and/or produce data in a particular format. For example, a particular connection pin factory 
may support a number of different data formats, such as 16 bit 44 kilohertz PCM audio or 
8 bit 22 kilohertz PCM audio. As explained previously, the connection pin factories and 
their different capabilities such as data format can be queried from the filter using the 
appropriate property set mechanism and the system I/O facilities. Actual connection pin 
instances are created based on the information received from the pin factories. 
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In a streaming environment, where a single stream write or stream read operation 
from a user mode agent will cause successive processing of the data through the connected 
filters, two main methods fox IRP control can be used as part of the native facilities of the 
NT operating system. First, a separate [RP may be created by each filter and sent to the next 
filter for processing which will in turn create a new IRP for further processing down the 
chain. The other method is to use a single IRP and pass it between the successive filters 
using standard procedures provided for interacting with the I/O manager. If the first method 
of creating new IRPs for each successive filter in the chain is used, interconnection order 
between the filters is ujiunportant since the filter need only know the destination of the IRP 
in order to call the I/O manager and send the IRP to the designated filter. If an IRP is reused, 
it is important that the connection pin instance transitions from the stop state be made 
beginning from the last filter to receive the reused IRP for processing backwards up to the 
first filter to receive the reused IRP or to the filter that created the IRP for processing. 

The current embodiment and implementation of the interconnected kernel mode 
filters utilizes ERP sharing advantageously to case complexity in driver development, allow 
more robust drivers to be created, and provide more efficient processing. The "bottom up" 
pin instance state transition path will ensure that the proper stack order is created in the IRP 
processed by the successive drivers and that each driver object has the appropriate stack 
depth parameter set. Furthermore, the current state of the receiving input pin instance is 
checked in order to assure that the state transition sequence has been properly followed. For 
this reason, the communications property of a particular connection pin factory will 
determine the potential flow direction and aid in properly distributing the state transition of 
connection pin instances. 

When creating an output pin instance (or IRP source), a reference to a file object 
representing an input pin instance (or IRP sink) on another filter will be passed as part of the 
NtCreateFile call. The appropriate create handler will be executed as explained previously 
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using the multiplexing dispatch function and device object/file object hierarchy. This create 
handler will have access to the device object of the filter having the input pin instance (e.g., 
Filter B 148 in Figure 7) by way of the input connection pin instance file object (e.g., input 
pin instance 154). From the device object, the previous stack depth parameter can be.read, 
and the stack depth parameter of the device object for the filter having the output pin instance 
may be incremented. For example, the device object associated with Filter A 1 46 will have 
a stack depth parameter incremented from that of the device object associated with 
Filter B 148 for the connection illustrated in Figure 7. This normally occurs when 
transitioning out of the stop state and IRPs are not forwarded while a connection pin instance 
is in the stop state. 

When a filter processes an IRP, it knows which stack frame or location within the 
IRP stack to access containing information designated for that particular filter by making 
reference to and using the stack depth parameter of the associated device object. 
Furthermore, the current filter will prepare the IRP for the next filter in the processing chain 
by decrementing the device object stack depth parameter to locate the next filters IRP stack 
location. 

The filter code is responsible for preparing the next location in the IRP stack and for 
calling the I/O manager to pass the IRP to the next filter as designated. In this manner, the 
filter may designate which file object representing a particular connection pin instance is to 
receive the IRP and the associated data for processing. Hence, the standard I/O manager 
calls such as IoAttachDevice to stack the respective device objects for sequential processing 
of IRPs are not used. 

It is noteworthy that creating a connection between connection pin instances does 
not imply creating new device objects to represent the connection. A single underlying 
device object is used to support an instance of a filter and all connection pin instances on that 
filter. Specific information necessary for proper data processing is kept within the context 



(104) 10-283199 

area of the file object allowing the context information to be preserved while non-page 
memory use is kept at a minimum. II is also noteworthy that while an IRP-based medium 
has been illustrated, other mediums for communication between the interconnected fillers 
may be used, such as direct function calls on non-host hardware-to-hardware communication. 

Referring now to Figures 9 - 1 0 and Figure I , the proper creation, connection, and 
state transition order of the software drivers as shown in Figure I (prior art) and Figure 2 
(higher level logical diagram of the interconnected kernel mode drivers) arc presented. 
Figure 9 illustrates the logical structure encompassed by box 162 and the processing steps 
contained therein. Figure 1 0 shows the creation of the connection pin instances to complete 
the interconnection of kernel mode filters and comprises the processing steps encompassed 
by box 164 on the flow chart shown in Figure 1 1 . 

When in the state of Figure 1 0 , having all interconnections made, the kernel mode 
filter system is ready for reads and writes in order to effectuate processing. The I/O system 
will use the IRP stack information properly set by the correct state transition process in order 
to pass the stream reads and writes onto the differing filter elements by way of their 
respective connection pin instances. It may be noted that some external software other than 
the agent used to create the graph, including a bridge or filter itself, as well as hardware will 
provide data for the stream reads and rights. 

After beginning at step 1 68, the controlling agent 1 70 will create instances of reader 
filter 172, decompressor filter 174, effects filter 176, and sound rendering filter 178 at 
step 180. Furthermore, attachment will be made between reader filter 172 and a disk 
driver 1 82 in order to bring the data in from off of the disk drive. Creation of each Filter 
instance is achieved by the user mode controlling agent 170 by using standard I/O calls to 
open a file on the appropriate device as found in the device I/O directory hierarchy. Such 
a call will return a handle to a file object representing the instance of each filter. 
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At step 1 84, the third party agent will query the effects filter 172, the decompressor 
filter 174, the effects filter 176, and the sound rendering filter 178 to determine the 
connection pin factory capabilities. These capabilities include what kinds of input and output 
pin instances may be created, how many instances of each connection pin factory the 
particular filter will support, the data format supported on each connection pin factory, the 
medium or type of communication path, etc. The capabilities are "queried" using the 
property set mechanism explained in more detail previously and the kernel mode filters are 
presumed to be compliant to the architecture since they support appropriate "sets" (eg., 
property set). 

All such query information at step 184 will be used to determine if a chained 
connection path is possible between the respective filters by creating and connecting the 
appropriate connection pin instances. The third party agent will determine the types of pin 
instances needed for interconnection in order to make a filter graph to accomplish a given 
purpose. 

The determination of the connection format based on the supported data formats is 
determined at step 186. Using the topology information, data format, and data intersection 
properties on the filter, a hypothetical filter graph may be created. Since connection order 
is not significant, this need not be done but could save time when trying to build a filter 
graph. Should this hypothetical filter graph be created without error, the third party agent 
will be assured that creating an interconnecting connection pin instances can be done with 
reliability. Because some queries will return errors unless an actual pin instance is created, 
it may be necessary to create such connection pin instances before a hypothetical filter graph 
can be created that will return a reliable indication of viability. Again, the hypothetical filter 
graph may be tested before any interconnections take place. 

Once the correct connection infonnation is known, as determined at step 186, the 
input pin instances may be created and interconnected and as represented by the loop of 



(106) 



ftmW- 1 0-2 8 3 1 9 9 



processing steps enclosed by box 1 64 on Figure 10. This loop contains processing steps that 
will begin at the input pin instance furthest away from the source of the data stream. This 
last input pin instance is referred to as the "deepest" pin instance and may be created first, 
followed by the output pin instance associated therewith. A connection, therefore, is the 
creation of an output pin instance using the handle of a previously created input pin instance. 

The pattern continues, with every input pin instance created successively afterwards 
prior to connection with the associated output pin instance. Such a connection scenario is 
givon by way of example and is not to be limiting of other possible ways of connecting the 
respective output and input pin instances to form a connection between kernel mode filters 
according to the present system. The filters may be connected in any order according to 
implementation as long as the handle from the input pin instance is used during creation of 
the connected output pin instance on another filter. Furthermore, as explained previously, 
changes may be made to the filter graph after initial creation (and even use). 

In the first iteration of the loop, input pin instance 1 88 will be created at step 190. 
After receiving the handle fiom the create function, the third party controlling agent 1 70 will 
use that handle as a parameter in an NtCreateFile call in order to create output pin 
instance 192 at step 194. By doing this through the first iteration, the sound rendering 
filter 178 is effectively connected to the effects filter 176 through the corresponding 
connection pin instances 188 and 192, respectively. In the current rniplementation, the 
NtCreateFile call is •Strapped" as part of a function call in an API made available to the user 
mode clients, This relieves the user mode developer of third party agents from needing to 
know as much detail and allows all relevant functionality be concentrated in a single user 
mode API. 

At step 196, the third party agent determines if there is any other ejdsting input pin 
instances to be created. If there are, an input pin instance must be created followed by the 
corresponding output pin instance on another filter. Eventually, all connections will be made 
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and the third party controlling agent 170 will prepare the filter graph for streamed data 
processing. 

In this fashion, input pin instance 202 will be created on the second iteration of the 
loop enclosed in box 164 at step 190 while the output pin instance 204 will use the handle 
of input pin instance 202 as part of its creation at step 1 94. Finally, on the third and final 
iteration for this particular example, input pin instance 206 will be created followed by 
output pin instance 208 to finalize the connection. 

At step 1 97, the third party controlling agent 1 70 will transition each connection pin 
instance from the stop state to the acquire state in preparation for streamed data processing 
through the filter graph. To correctly set the stack depth parameter in each of the device 
objects for the respective filters, it is necessary to make the state transition beginning with 
the "deepest" or last connection pin instance (e.g., the last input pin instance to receive data 
for processing) and sequentially moving "up" the chain of interconnected kernel mode filters 
until arriving at the first connection pin instance (e.g., the first output pin instance that will 
provide data into the graph). The first filter or bridge will create the IRP with enough stack 
locations allocated so that the IRP may be passed successively through each kernel mode 
filter in the graph in an efficient manner. 

Finally, the third part controlling agent 170 issues the stream reads and writes in 
order to process the data at step 1 98 before ending at step 200. 

As explained previously, each creation of an output pin instance will require the 
handle of a file object representing the input pin instance to be connected thereto. This file 
object reference will allow the create handler for the output pin instance to save a reference 
to the device object corresponding to the input pin instance for current or future access. 

More particularly, this allows the stack depth parameter of the device object 
managing the input pin instance to be accessed by the driver of the output pin instance during 
state transition from the stop state to the acquire or other state. The value of the stack depth 
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parameter associated with the input pin instance is accessed, incremented, and saved into the 
stack depth parameter for the device object corresponding to the output pin instance. 

The stack depth parameter is used to determine where in the shared IRP slack 
structure the stack frame information is located for a particular fdter and will be different for 
each filter. By so interconnecting the filters and making the state transition in proper 
sequence, a single IRP may be passed down the chain of interconnected filters in kernel 
mode with no necessary communication into user mode. 

It may be noted that it is possible to have multiple instances based on the same 
connection pin factory. For example, an audio mixing filter may mix multiple input pin 
instances into a single output pin instance in terms of processing. Each input instance is of 
the same type and the fdter may only support one type of input pin. Such an arrangement 
would also be an example of having multiple inputs to a single output. 

The converse is also true wherein a splitter filter may have a single input connection 
pin instance while providing multiple output pin instances thereby multiplying streams of 
data Those skilled in the art will note that many variations and useful combinations can be 
made from the connection mechanism explained herein according to actual implementation 
and the needs thereof. 

The uniformity and standardization achieved by requiring all compliant fdtcrs to 
support a common mechanism (e.g., property sets, methods sets, and event sets) that can be 
independently implemented by driver developers allows a controlling agent to conveniently 
connect compliant filters provided by various different software providers. Furthermore, 
many of the fecilities in terms of connection pin factories needed in one circumstance may 
not be needed in another circumstance. A determination of the necessary connection pin 
instances is made initially by the third party controlling agent that makes the actual 
interconnections between different filters. 
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Referring now to Figure 1 1 A, operation of a buffer allocator mechanism is shown 
as used between multiple processing components. Atso shown is the data flow (La actual 
data transfers between buffer frames) between buffers associated with particular processing 
components. While control will pass to each processing component, data will be transferred 
only when necessary with some processing components performing their data manipulations 
and returning the data to the existing buffer frame. In other words, data may be processed 
in the same location without being transferred to a new buffer and is said to be processed "in 
pi ace 

A sink processing component 210 will have a buffer allocator mechanism 212 
(represented by a square) as part of its functionality. A buffer allocator mechanism will only 
be present in processing components where it is necessary to ensure certain placement of the 
data into specific memory such as on board memory on a sound or video processing card or 
where the previous buffer has unacceptable characteristics, such as byte alignment, frame 
size, etc. Furthermore, references to the buffer allocator mechanism to be used for allocating 
frames of buffer memory is indicated by a circle and all processing components will have 
such references. Note that the source processing component 214 has a buffer allocator 
reference 216 that references the sink buffer allocator 212 as represented by arrow 218. 
Furthermore, the transfer processing component 220 has a null buffer allocator reference 222 
and the sink processing component 210 also has a null buffer allocator reference as indicated 
by the empty circle 223. 

In this simple processing example, the source processing component 214 allocates 
a buffer frame 224a by using the sink buffer allocator 212 and accessed using buffer allocator 
reference 216. The allocated frame 224a is filled with data by the source processing 
component 214 as represented by arrow 226. It may be noted that the source processing 
component may perform some data manipulation or transformation before writing the data 
into the newly allocated frame 224a. 
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At this point, the source processing component 214 has finished processing and 
turns control of processing to the transform processing component 220 as represented by 
arrow 228. The transform processing component 220 will do no buffer allocation or 
transferring of data from one buffer to another since no bufTer allocator mechanism is 
indicated since buffer allocator reference 222 reference has a null value. Therefore, the 
transform processing component 220 performs an "in place" data transformation in the 
allocated buffer frame 224b as represented by arrow 230. 

Since the data has not been transferred to a new buffer frame, buffer frame 224a, 
frame 224b, and frame 224c are the same frame and are simply passed in succession to 
different processing components. Arrow 23 1 represents passing the allocated frame between 
source processing component 214 and the transform processing component 220. 

Finally, the transform processing component transfers control of processing to the 
sink processing component 2 1 0 as represented by arrow 232. It may be noted that along with 
processing control, the same frame is passed for processing as shown by arrow 234 between 
the frame 224b and 224c. Again, as depicted herein, frame 224a, frame 224b, and 
frame 224c arc all the same frame allocated originally by source processing component 2 1 4 
and arc shown separately for illustration purposes. 

The sink processing component 210 will finish processing the data and free the 
allocated frame 224c within a buffer as presented by arrow 236. Since the sink component 
210 is no longer using the buffer, the arrow 236 points inward to the sink processing 
component 210 and the frame may now be deallocated or reused. 

Figure 13 shows how a bufTer allocator mechanism would be logically 
implemented in the scheme of interconnected kernel mode buffers that has been explained 
previously. Figures 12 and 1 3 both represent the same filter graph topology and are used 
to better illustrate operation of the bufTer allocator mechanism. The relevant drivers and 
portions thereof each have access points that allow user mode clients to control the driver and 
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are represented by file objects. I nlerconimuni cation occurs using IRPs (whether created by 
kernel mode drivers or by the NT executive in response to user mode I/O operations). 

An instance of source processing component 23S (represented by a file object) has 
associated therewith an output pin instance 240 (also represented by a tile object) that 
provides a source of connection to another filter instance. An input pin instance 242 that is 
a "child" of a transform filter 244 was created having a reference to the output pin instance 
240 in the manner explained previously in more detail. In like manner, a sink filter 246 
having an input pin instance 248 is connected to an output pin instance 250, that is related 
to the transform processing component 244. 

In the system of interconnected kernel mode software drivers, a buffer allocation 
mechanism is related to input pin instances and is said to be created or formed on the input 
pin instance. Furthermore, output pin instances will logically have reference to a buffer 
allocation mechanism, if necessary, and the filter having the outpui pin instance will utilize 
that reference to make any buffer frame allocations and data transfer to new frames prior to 
turning control to another processing component As explained, a null reference will indicate 
that no data transfer to a new frame is necessary arid that processing may occur in the 
existing fame, (i.e. , after processing, the data is returned to the same buffer frame). Whether 
a buffer allocator reference is present is determined by the initial negotiation of the third 
party controlling agent that created the filter graph. 

The buffer allocator mechanism formed on an input pin instance 248 is represented 
by a file object while the dashed line 254 shows that the output pin instance 240 has 
reference (e.g. , a pointer or handle) to the file object representing the buffer allocator 252. 
In the example shown in Figure 1 3 , frames of memory are allocated from system memory 
256. 

Since the filters may be interconnected in a number of different ways, a buffer 
allocator though available, may not be necessary. A file object representing an instance of 
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a buffer allocator will only be created should the third-party controlling agent 
interconnecting the filters determine that it is necessary. In this manner, filters may flexibly 
be connected in a variety of configurations and still maintain optimal data transfer 
characteristics. Furthermore, default system buffer allocators can be provided to further 
reduce the development effort of driver developers. 

A third-party controlling agent will also query buffer requirements of connection 
pins as part of creating a hypothetical model before making actual filter connections. While 
some implementations may allow queries before pin instantiation, the present embodiment 
requires that the actual connection pm instance be created before the buffering requirements 
can be ascertained. Furthermore, the exemplary embodiment disclosed throughout is queried 
through use of the set mechanisms explained previously. 

When a third party client or controlling agent completes the interconnections of 
kernel mode filters to make a filter graph, it then initiates negotiation of allocator 
requirements using the handles of the input pin instances (or sink pin instances). By 
convention, the input pin instances define bufTer allocation requirements and provide a buffer 
allocation mechanism while the output pin instances have reference to any appropriate buffer 
allocation mechanisms associated with the input pin instances. Those skilled in the art will 
understand that other conventions may be used to effectively accomplish the same results 
including reversing the buffer allocation responsibilities between the input and output pin 
instances. 

Buffer allocation requirements are ascertained by using a particular property set 
mechanism that will be supported by all compliant filters. It may be noted that the "buffer 
allocator" property set may be organized in a number of different variations as may any other 
set. For example, the property set may have a single property wherein the property is a data 
structure having segmented information or the property set may have multiple properties, one 
for each distinct framing requirement element. A single property composed of a data 



structure is more efficient in sonic circumstances since fewer propeny set queries are 
necessary by the third party controlling agent in order to retrieve all the framing requirement 
information. Furthermore, a single data structure could be used not only to query 
requirement information but also for specifying parameters when an actual buffer allocator 
is created. 

The table below shows a non-exhaustive list of framing requirement elements that 
may be included in a data structure or as individual properties. Also included is an 
explanation of how such an element would be used in an exemplary embodiment. 
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TABLE 3 



Buffer Allocator Framing Requirement Elements 


Element 


Description 


Control Options (Create) 


This element contains control options that are specified during 
buffer allocator creation on a given connection pin instance. Some 
options include: 

System Memory: This control option indicates the use of system 

mCTTinrv for hllfiipT* rrRirv nllrtpji't'irtnc \1/H»n cnomHar) +Wa l_ , . irx~ _ 
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allocator must allocate memory from the pool (as specified in the 
allocation pool type element) located in system memory. If this i 
control option is not specified, it is assumed that the input 
connection pin instance (or sink pin) provides a system address 
mapping to on-board RAM or other form of storage on a device 
attached to the system. Such a device may be an add in card, a 
PCMCIA card, etc. 

Downstream Compatibility: This control option specifies that the 
allocator framing elements of the buffer allocator being created are 
compatible with the downstream allocator. This option is 
normally specified when an in place modifier is assigned an 
allocator for copy buffers. If the filter is not required to modify a 
given frame, it may submit the frame to the downstream filter 
without allocating an additional frame from the downstream 
allocator . 



(114) 



ftfflW- 1 0-2 83 1 9 9 



Requirements (Query) 


This clement contains the allocator requirements for a connection 
pin instance (input or sink) that are returned during the query of a 
connection point. Some potential requirements include: 

Jn-placc: This indicates that a connection pin instance has 
indicated that the Alter may perform an in-place modification. 
Otherwise, a buffer must be allocated for receiving the modified 
data. 

System Memory: The connection pin instance (input or sink) 
requires system memory for all buffer frame allocations. If this 
requirement is not set, it is then assumed that the connection pin 
instance provides a system address mapping to on-board RAM or 
other form of storage on a physical device. 

Frame Integrity: This requirement indicates that the connection 
pin instance requires mat downstream filters maintain the data 
integrity of specified frames. This may be used when statistics are 
required of a previous frame in order to properly process the 
current frame. 

Allocation Required: This requirement mandates thai connection 
point must allocate any frames sent. 

Preference Only Flag: This requirements flag indicates that the 
other requirements signalled are preferences only. This means 
that the requirements indicate a preferred way of operation but the 
connection point will still operate correctly and process frames 
which do not meet the specified requirements. 


Allocation Pool Type 


This element determines the allocation pool type from which 
kernel mode entities will receive system memory for buffer frame 
allocation. 


Outstanding Frames 


This element indicates the total number of allowable outstanding 
buffer frames that can be allocated In this manner, the total size 
of the buffer can be controlled- Furthermore, specifying a zero for 
this element indicates that the filter (through a particular 
connection pin instance) has no requirement for limiting the 
number of outstanding frames. 


rramc aize 


This element specifies the total size of a buffer frame. Again, 
specifying a aero for this element indicates that the filter (through 
a particular connection pin instance) has no requirements for this 
member. 


File Alignment ' 


This dement specifies the alignment specification for allocating 
frames on certain boundaries. For example, an octal byte 
alignment may be specified Furthermore, in the exemplary 
embodiment, the minimum alignment capability is on quad 
boundaries providing optimal data access speed on current PC 
hardware architectures. 
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Those skilled in the art will appreciate that there may be other relevant framing 
properties that may be included. Furthermore, the buffer allocation mechanism as described 
herein functions in substantially the same manner regardless of whether more buffer frame 
elements than specified in Table 3 are included or whether a subset of those specified are 
implemented. 

When the filter graph requirements have been determined by the third party 
controlling agent, then the appropriate kernel mode buffer allocators may be created on the 
appropriate input connection pin instances. The client creates a buffer allocator instance 
using the handle of the appropriate connection pin instance and by specifying the appropriate 
creation parameters for the buffer allocator. In this manner, the resulting file object that 
represents a buffer allocator will be a child of the connection pin instance and will use the 
context information from that file object and the file object representing the instance of the 
filter itself for its proper creation. 

In other words, the same mechanism explained previously for validating creation of 
a connection pin instance and for routing messages to the appropriate handlers for a given 
connection pin instance will apply in like manner to the creation of an instance of a buffer 
allocator. Again, the NtCreateFile call will be wrapped in a higher level function call as part 
of an API that third party controlling agent may use. 

Buffer allocator instances will be created, if at all, only on input pin instances in the 
presently explained embodiment. 

If a buffer allocator instance handle is not specified to an output connection pin 
instance handle through an appropriate API call, then the filter can assume that the stream 
segment submitted via stream I/O controls meet the filter's requirements and it may freely 
modify the data in-place. 
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A system supplied default allocator may be used by filter developers to simplify the 
task of providing buffer allocation capabilities to input connection pin instances. The default 
allocator provides for a system memory buffer frame allocation for those device drivers that 
are able to transfer data from system memory but require specific memory allocation 
properties. By using the default buffer allocator, a filter developer is relieved from the task 
of actually providing code to do the buffer allocation. The filler will still be written, 
however, to support the buffer allocation requirements request by supporting an appropriate 
property set 

To use the default allocator, the filter designer will (I) provide code to respond to 
the buffer allocation requirements requests, and (2) place a default allocator creation handler 
reference in the validation table of the particular connection pin instance to which the default 
allocator pertains. In other words, when a create request for the allocator type mechanism 
is submitted through the filter, a specific GU1D string is matched in the validation table that 
in turn allows access to the default allocator creation handler. 

The default buffer allocator uses system memory and will operate in accordance with 
the buffer allocator framing properties submitted as part of the creation request. Such a 
default allocator is likely to be used by various transform filters depending upon their 
particular function and interconnection order. 

Fitters requiring buffer allocators for on-board memory or other device dependent 
storage methods can provide a specific buffer allocator by supporting a buffer allocator 
property set and method set For a filter specific allocator, the filter will be responsible for 
providing program code to implement the entire functionality. The operation of the buffer 
allocator, however, will be the same as for every buffer allocator, whether default or filter 
specific, so third party agents may interconnect the filters and buffer allocators properly. 

The file object representing the buffer allocator, when successfully created, has 
contained in the file context area a pointer to a data structure. This data structure will contain 
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a dispatch table for directing IRPs to designated handlers based on the various IRP codes 
(e.g. create, I/O control, etc.) as well as other data areas and structures for maintaining the 
state of the buffer allocator. Some implementation dependent information that may be 
tracked includes reference to the fdc object of the connection pin instance to wtuch the buffer 
allocator pertains, reference to an allocation framing requirements data structure, an event 
queue of those clients waiting for events, a queue of those allocation requests (received by 
an IRP, for example) that are outstanding, etc. 

There are two interfaces, or ways of communicating, available to the buffer 
allocation mechanism disclosed herein in the exemplary embodiment. First, all allocators 
must provide the [RP-bascd interface in order to communicate properly with user mode 
clients. Optionally, if the allocation pool type can be serviced at the dispatch level (a raised 
priority level al which a limited subset of services are available, but at which lower priority 
tasks are blocked out on that processor) of the operating system, a function table interface 
may be supported allowing interconnected filters to use direct tunotion calls (during DPC 
processing) for higher performance. This allows event notifications to be communicated 
between the interconnected filters without the extra overhead of passing an 1 RP through the 
I/O manager, or scheduling a thread of execution and waiting for a context switch. 

the IRP -based interface will successively process IRPs in the following fashion. 
When an allocate request is submitted, the buffer allocator will complete the request and 
return a pointer to an allocated buflfer frame or if all frames have been allocated, the allocator 
marks the IRP pending and adds the IRP to the allocator's request queue for processing, in 
approximately FIFO order. Finally, the buffer allocator will return status pending to the 
caller. 

When a buffer frame is made available to the allocator, the allocator attempts to * 
complete the first request in the request queue the for IRP-bascd interface. Those IRPs that 
could not be serviced previously will be in this request queue and will be waiting for the 
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newly freed frame for processing. Otherwise, if no work awaits in the request queue, the 
frame is returned to the free list. 

The dispatch level interface using the direct function call table operates in the 
following fashion. When an allocate request is submitted by making a function call (which 
may be done during a DPC), the allocator returns a pointer to the frame if one is available 
or returns null immediately. The kernel mode requester may then wait for a free event 
notification to know that there is a free frame available. Upon receipt of this notification, the 
kernel mode requester will attempt the allocate request again. 

Note that it is possible for bo Hi the dispatch level interface and the IRP-based 
interface to contend for an available free frame. Also, if there are allocation request IRPs 
waiting to be completed in the request queue, the allocator must schedule a worker item if 
the current IRQL is not at the passive level when the caller frees a frame since using the 
direct call interface implies the possibility of being at the DPC level. Essentially, the 
waiting queue of IRPs does not find out about the free frame until the worker Item has been 
run. 

Furthermore, the buffer allocation mechanism explained herein is adaptable to be 
used with MDLs (memory descriptor lists) in order to further reduce inter buffer transfers. 
Such an implementation would allow greater seamless interaction with the system memory 
facilities of the NT operating system. 

Referring now to Figure 14, the interconnected filter system of Figures 9 and 1 0 
are shown utilizing the buffer allocation mechanism disclosed previously. Once the 
controlling agent 170 has made interconnections between the filters, it will query each input 
pin instance for its buffer requirements. As shown herein, the sound rendering filter 1 78 will 
need to allocate buffer frames from the sound card memory 258 and the decompressor filter 
174 will allocate memory from system memory 260 which may receive the data directly 
from disk drive 262. 
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Controlling agent 170 will create a buffer allocator 264 thai is represented by a file 
object and being formed on the input pin instance 1 88. The buffer allocator 264 will allocate 
buffer frames from the sound card memory 258 and a reference to the buffer allocator 264 
wUl be set in the output pin instance 204 as represented by dashed line 266. This reference 
will be the handle to the file object representing buffer allocator 264 and will be used by the 
decompressor filter 1 74 to allocate bufTer frames as necessary before transferring data to the 
new buffers. 

Ixi like manner, buffer allocator 268 is also represented by a file object and created 
or formed on input pin instance 206. This buffer allocator 268 will manage allocation of 
system memory 260. The controlling agent 170, after creating buffer allocator 268, will 
store a reference thereof in output pin instance 208 as represented by dashed Line 270. 
Again, buffer allocator 268 will be responsible for allocating bufTer frames between system 
memory 260 so that data may be transferred therein from disk 262. 

The controlling agent will also put a value of null for the buffer allocator reference 
of output pin instance 192 to thereby indicate that an in place transformation will occur 
wherein the effects filter 176 will read the data from the existing buffer in sound card 
memory 258 and replace the data back into the sound card memory 258 after applying 
whatever transform or effect is required. Alternatively, if the controlling agent does not set 
the bufTer allocator reference, it will be assumed to have a value of null and an in place 
transformation, will occur. 

Referring now to the flow chart of Figure I S and the logical diagram of Figure 14, 
operation of the buffer allocators will be shown. This process occurs after the 
interconnections are made and the stream reads and writes are given from the controlling 
agent 170 to the reader filter 172. 

initially, the reader filter 172 allocates a frame in the system memory using the 
decompressor filter 174 buffer allocator 268 at step 272. The oulput pin instance 208 of 
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reader filter 1 72 will have a handle to the file object representing the buffer allocator 268 as 
represented by line 270, and will therefore have direct access to controlling the buffer 
allocator 268. 

Once the file reader filter 1 72 has access lo an actual buffer frame in system memory 
260, it will fill the frame with data from disk 262 as represented by arrow 274 at step 276. 
It may be noted that the reader filter 1 72 may make a transformation or other manipulation 
on the data as it brings it into system memory 260. 

J he file reader filter 1 72 then initiates a stream write to the decompressor filter 174 
at step 278. This stream write will be passed by way of an 1RP through the NT VO manager. 
At step 280. the decompressor filter 174 will allocate a frame of sound card memory 258 
using the buffer allocator 264. The decompressor filter 174 will have knowledge of the 
buffer allocator 264 since a handle thereto was stored with respect to output pin instance 204. 

The decompressor filter 174 will decompress the data and transfer the data to the 
frame previously allocated in sound card memory 258 as represented by arrow 284. Note 
that in decompressing the data there may be more frames allocated from sound card memory 
than exist in system memory. The extra ratio of buffer frames is necessary to accommodate 
the decompression effect on the data. 

It is important to note that when transferring data from one buffer to another buffer 
that there may not be a 1 : ] correspondence in terms of the amount of data transferred. In 
other words, the receiving buffer may require more or less space (or frames) depending on 
the nature of the filtering or transformanon that takes place between the buffer transfer. 

Once the decompressor filter 174 is finished decompressing a particular frame, it 
passes the stream write to the effects filter 1 76 using the facilities of the NT I/O manager. 
The effects filter 176 receives the stream write IRP at step 288 and processes the data in 
place in the existing sound card memory 258. Such effects processing would generally 
correspond to a 1 : 1 replacement of data requiring neither more or less buffer memory. 
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Once the effects filter 176 is finished processing the data in place, the stream write 
IRP is passed to the sound rendering filter at step 290. Again, the mechanism causing the 
stream write JRP to be transferred to the sound rendering filter 178 is the NT I/O manager 
facilities called by the effects filter 176. 

Finally, the sound rendering filter 178 will receive the stream write IRP at step 292 
and control the sound card hardware in order to form the actual rendering of the sound data 
existing in sound card memory 258. At that point, the sound card buffer frames that had 
been previously allocated can be reused to service writing requests or freed if there are no 
outstanding requests. The availability of a buffer frame is made known to the decompressor 
filter 174 so that any waiting stream writes may be used to process and place data into the 
freely allocated buffer frames. In like mannei, the buffer frames of system memory 260 are 
either reused or freed. 

Referring now to Figure 16, an overview system is shown wherein a master clock 
mechanism can be used to synchronize two data streams that are generated in real time and 
said to be 'live." An audio renderer 296 receives a live audio stream 298 for processing as 
represented by arrow 300. As the audio renderer 296 processes the data, it is perceived as 
sound through a speaker 302 once the audio renderer 296 has converted the data samples into 
the appropriate electronic signals and sends the signals to the speaker 302 as represented by 
arrow 304. 

In like manner, a video renderer 306 receives a live video stream 308 as represented 
by arrow 3 1 0 for processing. As a result of being processed by the video renderer 306, the 
data samples are converted into appropriate video signals and are transmitted to a monitor 
312 as represented by arrow 314 where they may be perceived as images. 

Typically, the live video stream 308 and the live audio stream 298 will come 
simultaneously into the rendering components. Since it is important to match the video with 
the audio when rendered at the speaker 302 and the monitor 312 in order to produce a 
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coherent presentation, processing of the live video stream 308 and the live audio stream 298 
must be synchronized. 

In order to accomplish this, one data stream is chosen to be the master. For 
purposes of this disclosure and explanation, the live audio stream 298 has been arbitrariJy 
chosen to be the roaster data stream. Note that nothing prevents the live video stream 308 
from being chosen as the master or any other stream of data for that matter. Furthermore, 
an external clock, based on a data stream entirely external to the filter graph, may be used 
to synchronize both the live audio stream 298 and the live video stream 308. Therefore, the 
master clock mechanism 3 1 6 is shown appended with a solid line to the audio renderer 296 
and has a dashed line 318 to the video renderer indicating that the video renderer will in 
some manner receive synchronization information from the roaster clock 3 1 6 that is using 
the audio stream as the master. 

The master clock 3 1 6 may also be used by the audio renderer 3 1 6 in order to match 
the rate of processing at the audio renderer 296 with the rate of the original audio stream 298. 
In other words, the live audio stream 298 was produced by another processing component, 
such as digitizing filter/digitizing hardware on another system. Since the audio stream is 
composed of digitized samples, the samples were produced at a rate according to the 
hardware and oscillator of an original processing component. This origination rate for the 
original processing component to produce the live audio stream 298 is found in or derived 
from the time interval information of the live audio stream 298. 

Sample information in a data stream may contain time interval infomiation in a 
variety of ways. One way is for each sample or group of samples to have a timestamp 
associated therewith. Furthermore, the format of the data itself may imply the sample rate. 
For example, by the definition of the data format, each sample may be taken at a certain time 
interval (e.g., l/30tb of a second for a video sample or l/22,050th of a second for audio data). 
Additionally, a timestamp may indicate a presentation time that is matched to a clock at the 
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rendering processing unit to determine when a particular sample is to be rendered or 
"presented," Also, a timestamp may indicate some other time value with the relative 
difference between time stamps giving time interval information. Finally, mixtures of data 
formal and timestamp information may be used in order to provide the time interval 
information necessary for creating a time clock based on the position within a reference data 
slrearn being processed. . 

It may be noted that "discontinuities' 1 may be coded into the data stream that may 
be based on time or data gaps. A time discontinuity is where the stream simply jumps to 
another presentation time, such as when recording ended at a certain time and suddenly 
began at a later time. A data discontinuity is where the stream may have periods of repeated 
data that can be more efficiently coded in the stream, such as representing periods of silence 
by a given code followed by duration of the silence. 

One time value provided by the master clock 3 16 is based on the time interval 
information found in a stream of data samples and known as media time. Media time is 
positional since it represents a position within the master stream, which in this case has been 
chosen as the live audio stream 298. Furthermore, when processing quits at the audio 
renderer 296, the media time is also frozen at the master clock 316. A discontinuity in the 
data stream will be processed to that "position" within the stream continues to advance while 
in the discontinuity by using the underlying hardware oscillator or other source to track the 
advancement of time. In this manner, multiple data streams may still be synchronized to a 
data stream having discontinuities of time or therein. 

By synchronizing the video renderer 306 to the media time found on the master 
clock 316, which in turn is based on the time interval information in the live audio 
stream 298, no overhead is necessary to manage stopped or paused states since media time 
advances (''ticks") only when the live audio stream 298 is being processed at the audio 
renderer 296. 
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It should be noted that the video stream 308 may also need to be rate matched as 
well as synchronized. The rate matching of the video stream 308 would occur in much the 
same manner as will be explained hereafter in connection with the live audio stream 298 and 
the audio renderer 296. Naturally, the live video stream 308 rate matching is based on the 
hardware oscillator (physical time) of the video renderer 296. 

Also available at or otherwise provided by the master clock 316 is a physical time 
value that is based on the underlying hardware in the audio renderer 296. Physical time 
usually progresses regardless of whether the audio renderer 296 is actively processing audio 
data or is in a stopped or paused state. Furthermore, by sampling the rate of progression of 
the media time (representing the originating hardware processing rale) and the physical time 
(representing the actual processing rate at the audio renderer 296) adjustments may be made 
at the audio renderer to process the live audio data 298 so that the rates are relatively 
matched. 

While rate matching is not an important issue when the rate of data production is 
under control of the processor such as reading stream data from a local disk or other storage 
device, it is crucial when processing a "live" data stream that continuously produces data and 
the production rate is out of the control of the processor. As used herein, a "live" data or 
media stream refers to a stream produced at a rate that is not controlled by the processing 
components and, therefore, must be processed completely in real time or lost Examples of 
a live data stream include but are not limited to the following: a real time feed where the 
stream is produced in real time either locally or over a network connection, receiving the 
stream from a network connection regardless of whether it is a real time feed or previously 
stored data sent over the network, etc. In all cases, the stream must be processed as it is 
received. 

If the live media stream produces too much data for the audio renderer 296 to 
process, the audio renderer 296 will eventually lose data due to finite buffering capabilities 
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in a condition that is known as data "flooding." On the other hand, if the live audio stream 
298 produces data at a lower rate than the audio Tenderer 296, the audio rendcrer will suffer 
data "starvation" in that it will not have data to process. In either case, it is beneficial for 
quality rendering of the live audio stream 298 to make adjustments to match the rendering 
rate at the audio Tenderer 296 with the origination Tate as represented in the time interval 
information in the live audio stream 298- 

One way of making rate correcting adjustments is to periodically duplicate data 
samples so that the audio rendcrer will not become starved or to throw away samples so that 
the audio tenderer will catch up with the live audio stream 298 that outproduces its 
capabilities. In either case, duplication or omission of data samples would judiciously occur 
at intervals over stream processing so that a minimal amount of impact is perceived at the 
speaker 302. 

Rather than synchronize directly to the master clock 316, it may be useful to 
translate another clock's time base to match the media time found in master clock 3 1 6 or to 
translate presentation time information in another media stream to match the media time of 
the master clock 316. In order to facilitate such translation, the master clock 316 will 
provide a correlated media rime value that consists of two separate time values, the media 
time value and a reference time value common to other components, and does so in an 
atomic (e.g., unitary) operation to reduce the introduction of timing errors. A correlated 
physical time value is also provided. 

The reference time is typically based on the PC clock which should be available to 
all processing components or other clock mechanisms found on the system though another 
clock could be chosen. The atomic operation may make use of native system facilities for 
implementation so that both values may be retrieved with the least amount of interruption 
or other interference. The correlated time (cither media or physical) can be used by another 
processing component, such as a filter, to determine a relative offset or delta between the 
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media time and the reference time in order to make any necessary translation with a 
minimum of error introduced as will be explained in more detail hereafter. 

Referring now to Figures 17 and 1 8 , a clock mechanism as described above is 
implemented with the interconnected filter system explained previously. Figure 17 
represents a live video stream processing set of filters thai is syrK^ironized to an audio stream 
and receives synchronization information from a master clock mechanism. Figure 1 8 is a 
set of interconnected filters for rendering a live audio stream that is used to generate the 
master time reference for a master clock mechanism. 

Referring first to the video stream processing components in Figure 1 7 ., a live 
video stream 320 is received by a video reader filter 322 as represented by arrow 324. The 
source of the live video stream 320 may be from on-board digitizing hardware at the 
computer, packets of digitized samples received over a computer network, etc. The live 
video stream 320 is comprised of digitized samples that may be rendered into actual video 
images and the samples have time interval information associated therewith as explained 
previously. Again, according to the interconnected filter system explained herein, the video 
reader filter 322 is represented by a file object 

The video reader filter 322 has an output pin instance 326 that is connected to an 
input pin instance 328 on a video decompressor filter 330. Video decompressor filter 330 
will receive the live video data in compressed format and decompress it into a format 
acceptable to video rendering hardware. After processing frames of video data, the 
decompressor filter will turn control over to the video rendering filter 332. A connection is 
made between the video renderer filter 332 and the video decoir4>rc SS or filter 330 by means 
of the output pin instance 334 and the input pin instance 336, respectively. 

The input pin instance 336 will receive timing indications for synchronization from 
the master clock mechanism in the audio stream processing components (see Figure I 8 ) 
as represented by box 338 and dashed line 340. This synchronization timing information 
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may be received in a variety of different ways. First, a controlling agent 342 that 
interconnects the filter graph topology shown in Figures 1 5 A and 1 5B may liavc a clock 
mechanism 344 make event notifications based on either a specific position {i.e., time) withiu 
the master audio stream or a specific time interval may be chosen for interval event 
notifications. Alternatively, the video rendering filter 332 may query the clock mechanism 
344 (see Figure 1 8 ). Uecause the clock mechanism 344 is represented by a file object, the 
controlling agent 342 can make appropriate references to the clock mechanism 344 to the 
input pin instance 336 or the video renderer filter 332, thereby allowing the video renderer 
filter 332 to query the clock mechanism foT correlated time values in order to make 
translations, if necessary, or media time values to assure that processing is synchronized. 

Finally, the video renderer filter 332 controls the video hardware 346 and transfers 
data thereto as represented by arrow 348. The video hardware will render the data and 
generate video signals causing images to be displayed on monitor 350. 

Referring now to Figure 1 8 , the processing components of the live audio stream 
are now explained in detail. Again, the same controlling agent 342 will create the respective 
filter instances along with any nec es sary input pin instances and output pin instances in order 
to make die interconnections between the respective filters and it is the controlling agent that 
makes the entire kernel mode filter graph topology, 

The live audio stream 352 is initially brought into the filter graph topology by the 
audio reader filter 354 as represented by arrow 356. The data is decompressed by the audio 
decompressor filter 358 that is connected to the audio reader filter using respective output 
pin instance 360 and input pin instance 362. Again, each fitter and connection pin instance 
described herein is represented by a file object and created as explained previously. 

The audio decompressor filter is connected to the audio renderer filter 364 through 
the output pin instance 356 and the input pin instance 368, respectively. Also associated 
with the input pin instance 368 is an instance of a clock mechanism 344 that will provide or 
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send indications of media time, physical time, and/or correlated times to other entities, 
particularly the input pin instance 336 for the video Tenderer filter 332 (see Figure 1 7 ) as 
represented by box 370 and dashed line 372. The actual implementation and availability of 
the clock mechanism 344 will be explained in more detail hereafter. By using the clock 
mechanism 344, other data streams may synchronize processing to the processing of the 
"master" or "reference'' audio stream 352 as illustrated in Figure 18 

The audio tenderer filter 364 controls the audio hardware 374 and will pass control 
information and data as represented by arrow 376. The audio hardware, under the control 
of the audio Tenderer filter 364, will render the digitized data samples into real time 
electronic signals that may be transmitted to and perceived from a speaker 378. 

When creating interconnected kernel mode filters as explained previously, each filter 
instance is created from a "device" available to the system, in this manner, the video reader 
filter 322 and the audio reader filter 354 may be separate instances of the same file reader 
device. 

Additionally, because a particular filter may support many connection pin factories 
for creating connection pin instances, the video decompressor filter 330 and the audio 
decompressor may be instances of the same decompressor device. When the controlling 
agent creates connection pin instances on the respective filter instances, different pin 
factories are chosen as templates for creating the connection pin instances depending on the 
type of data being processed through the filter. This is the inherent flexibility in the 
interconnectable kernel mode filter system disclosed herein. 

In order to form clock mechanism 344, a controlling agent 342 will typically query 
a connection pin instance, such as input pin instance 368 on the audio renderer filter 364, to 
determine if a clock mechanism is available. Other implementations may require the 
controlling agent 342 to simply attempt creation of a clock mechanism on a connection pin 
instance, thereby determining clock mechanism availability by trial and error. Furthermore, 
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a clock mechanism may exist in user mode and send notifications, etc., to input pin instances 
of filters in kernel mode through a kernel mode proxy of that clock or some other 
mechanism. 

By chosen convention in the exemplary embodiment, timing event notifications and 
clock mechanisms are typically found in association with input connection pin instances. 
Clearly, however, any entity represented as a file ohject may receive event requests by way 
of IRPs or by a direct procedure call after retrieving the direct procedure call table through 
a previous IRP request. 

A system supplied default implementation of a clock mechanism may be supplied 
or made available to software filter developers in order to provide timing fiincb'onality 
without requiring additional code development Formation of any clock mechanism 
(supplied default or custom) occurs by specifying a GUID siring value in the create request 
that will be used in a connection pin instance validation table in order to access the correct 
creation handler for forming the clock mechanism on the connection pin instance. The 
process of file object creation, validation, and designation of IRPs was discussed in more 
detail previously in connection with Figures 3, 4A-4C, 5, and 6. In the same fashion as 
creating a connection pin instance, a clock mechanism create request may be properly routed 
and validated. 

In order to use the default clock supplied by the system, a driver developer will call 
the default clock create method from within the create handler that they create and provide 
to the system. The create handler is accessed from the context area of a file object 
representing a connection pin instance in the manner described previously. The driver 
developer, therefore, implements a driver specific clock mechanism, and the appropriate 
driver specific create handler would be placed in the validation table. Complexity of create 
handler development is reduced by making a call to the default clock create method rather 
than coding and implementing the clock functionality. 



For a clock mechanism to be made according to lhe interconnected filter system 
shown throughout, it must support a certain property set and event set. Naturally, the default 
clock mechanism supports the specified sets and any filter specific clock mechanism would 
require the filter developer to write and provide code to implement the clock mechanism 
property set and event set. In this manner, any controlling agent having a handle to a clock 
mechanism may query for support of the specified set and know how to operate the clock 
mechanism. Table 4, below, indicates a number of properties that may be included in a clock 
mechanism property set. 
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TABLE A 



Properties 



Media Time 



Physical Time 



Clock Mechanism Properties 



Description 



This property returns the current media time on the clock as 
represented in standard units. For example, in the exemplary 
embodiment, thu would be timed in 100 nanosecond units. In the 
common case where the clock mechanism is being presented bv 
some processing component, such as a filter or connection pin 
instance on a filter, which is processing a timestamped data or 
control stream, the media time value returned by this property 
reflects the current position in the stream. Furthermore, any data 
or control stream having time interval information may be used to 
allow the clock to properly operate. If the underlying processinc 
component becomes data starved, the media time as presented by 
this property also stalls. Furthermore, if the data stream or controL 
stream is comprised of ttmestamped data, and the umcstarnps on 
the stream are modified, the media time as presented by this 
property will also reflect this modi ficanon. Therefore if 
timestamp information is n tran5lated H from one time frame to 
another, any clock mechanisms based on that data stream will also 
be translated. Finally, if the underlying component is directed to 
stop or pause processing the data or control stream, the media time 
as reflected by this property will also stop. 



This property returns the current physical time for the clock 
mechanism. The physical time is a continuously runrung time 
wruch is based on some underlying hardware oscillator. For the 
default clock mechanism, this would be the PC clock. For a filter 
specific clock mechanism, it may be the actual rendering hardware 
clock. Physical tune does not stall when stream starvation occurs 
and may not stop when stream processing is stopped or paused bv 
the underlying processing component. The physical time rate of 
progression matches that of the underlying hardware and therefore 
allows an interested party to compare the rate of the physical time 
with the rate of its own processing tune or media time progression 
in order to match processing rates. In concert with querying- the 
current media time, and with feedback as to stream processing 
starvation or flooding, a processing component can not only make 
adjustments for matching processing rates, but also determine if it 
isariead or behmd a desired stream position fox the master or 
reference stream (i.e., synchronize). 
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Correlated Media Time 


Thus property returns both the current media time value for the 
clock mechanism, and a corresponding reference time value as a 
single atomic operation. In the exemplary embodiment, the PC 
time serves as a reference lime that is universally availahle to 
many different operational entities. By using the correlated media 
time, a > processing component using a different clock may perform 
a translation using the common PC lime while introducing a very 
small, and in many instances insignificant, amount of error. Since 
translations back and forth would use the same atomic operation, 
the small error is not cumulative. 


Correlated Physical Time 


This property returns both the current physical time value for the 
clock mechanism, and a corresponding reference time value as a 
single atomic operation. In the exemplary embodiment, the PC 
time serves as a reference rime that is universally available to 
many different operational entities. By using the correlated 
physical time, a processing component using a different clock may 
perform a translation using the common PC time while 
introducing a very small, and in many instances insignificant, 
amount of error. Since translations back and forth would use the 
same atomic operation, the small error is not cumulative. 


Granularity & Error 


This property returns the clock increment granularity or resolution 
in order to indicate how precise or fine each clock "tick" will 
appear. In the exemplary embodiment, the granularity value is the 
number of 1 00 nanosecond increments per each clock tick. If a 
clock runs at or under 100 nanosecond resolution or granularity, 
the value would simply return 1 . The granularity property allows 
a client of the clock to react differently depending on the 
resolution. The property further provides an error indication in 
100 nanosecond increments with 0 representing the least amount 
of error. 


Parent 


This property returns the unique identifier of the processing 
component which produced the clock mechanism. This allows a 
client or processing component to determine if the clock 
mechanism is its own clock mechanism or some other processing 
component's clock mechanism so that it may react differently as 
the circumstances dictate. For the exemplary embodiment, this 
property would be a handle to the file object of an input 
connection pin instance or filter upon which the clock mechanism 
was produced. 


Component State 


This property reflects the state of the underlying processing 
component (i.e., filter) so that a clock mechanism client may 
determine if the underlying processing component is in the stop 
state, the pause state, the run state, or the data acquisition state. 
Again, this property functions similar to the media time in that it is 
dependent upon what is happening to the underlying data stream 
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Function Table 



This property indicates a data structure or table that allows a client 
to access a table of functions which represent a subset of the 
property set interface. Of necessity, this property is platform 
specific since kernel mode function calls cross filter boundaries 
may be handled differently by different operating systems. In the 
exemplary embodiment, using the NT operating system, the 
function table allows other kernel mode entities to have a faster 
interfere available at the DPC level. This is contrasted with the 
normal method of using properties through NT kernel fRPs 



Those skilled in the art will appreciate that other properties than those listed in 
Table 4 or a subset of the properties listed in Table 4 may be used to make a clock 
mechanism property set that provides functionality according to the interconnected filter 
system explained herein. Furthermore, the flexible property set mechanism allows a filter 
developer with unique tuning capabilities to extend a "required" property set by adding 
thereto another property set, having its own GUED> for more advanced capabilities. The set 
mechanism allows the flexibility to filter developers in that a minimum amount of 
functionality may be implemented so that the filter is compliant with the system architecture 
while allowing unbounded ability for the filter developer to customize and add additional 
capabilities. 

While a property set allows other processing components, such as kernel mode 
filters, to query relevant time properties, clock mechanisms may also support an event set so 
that notifications may be sent directly to interested clock mechanism clients by way of event 
notification IRPs. The handle of a file object may be "registered" with the clock mechanism 
as part enabling an event so that the clock mechanism will know where to send event 
notifications. Optionally, direct function call handles may be used so that DPC level 
processing can occur between kernel mode entities for achieving higher performance. 

Table 5 below, a number of possible notification events make up an event set that 
can be supported by compliant drivers. A third parry controlling agent may interconnect or 
register those entities for receiving events based on e given filter graph topology. 
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Clock Mechanism Events 


Event 


Description 


Interval Event 


An event which is to be signalled in a cyclic manner. In the 
exemplary embodiment, a specific interval and specified clock 
rime are set by a controlling agent. Beginning at the specified 
clock time, cyclic event notifications will be sent at the ncfurrpnrj* 
of each interval. For example, if a processing component renders 
video frames from a video stream at thirty frames per second it 
would specify a start time and an interval of L/30th of a second so 
that the event notifications would be sent at the same frequency 
thereby allowing the video frames to be rendered properly. 


Positional Event 


An event which is to be signalled at a specified clock time. For 
example, a processing component or filter which renders a media 
event stream may have the need, in some circumstances, to play I 
notes at a specific clock time. The processing component or filter 
could specify a positional event notification to occur at the 
designated clock time so that it would know to begin rendering the 
note. Positional notification events could also be used to 
determine exactly when the clock begins to progress after being 
put into the run state by setting an event notification on the 
stopped or paused clock to be signalled at the current time. 



Those skilled in the art will appreciate that other timing related events may be 
created for a clock mechanism. For example, events may be broken down according to the 
different kinds of time (i.e., media time or physical time) that a clock mechanism may 
support Furthermore, "hybrid" events may be envisioned by combining elements of interval 
events and positional events, as well as other events based on state changes. 

Referring now to Figure WA„ a method for synchronizing one or more "slave" 
processing components or filters based on a "master" media processing time is presented. 
After starting at step 380, a slave processing component or filter receives the master clock 
media time value at step 382. Such receipt may occur by having the slave filter or 
component query the master clock mechanism for the media time value or the master clock 
mechanism may send notifications to the slave processing component or fitter. Applying the 
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method explained by the flow chart of Figure 1<?A to the filter graph topology shown in 
Figures ) 7 and 1 3 , the master clock mechanism 344 is represented by a file object which 
would communicate with the input pin instance 336 of the video renderer filter 332 as a slave 
processing component. 

Note also, thai the media time value used depends upon the master data stream 
processing. Should media rendering be paused or stopped, then time would effectively stop 
for synchronization purposes. 

The slave-processing component or filter will compare the master clock media time 
value to the slave data stream media time value at step 384 in order to determine how far 
ahead or behind the slave media processing has become. Note that media time is based upon 
the time interval information in the actual stream of data samples and therefor can be viewed 
as positional in nature. In other words, synchronization also means processing two data 
streams at the same relative time position within the stream. 

Before ending at step 388, the slave processing component or filter will adjust slave 
media processing to match the master media processing time at step 386. Such adjustments 
may include accelerating or decelerating slave media stream processing, changing the rate 
of the underlying hardware processor, etc. 

Decelerating media stream processing rate may be achieved by duplicating samples 
for processing, introducing mtermitterit time delays, or resampling. Accelerating media 
stream processing may be achieved by omitting media samples for processing, or resampling. 
Those skilled in the art will appreciate that numerous ways exist for adjusting the media 
processing rate once it is determined mat a change must be made. 

Note that in the example topok)gy srwwn m Figures n and 16 , input pin instance 
336 would receive timing notifications for the video renderer filter 332. The video renderer 
filter 332 would make whatever adjustments to processing that are necessary in order to 
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bring the processing of the video stream of Figure 17 . m synchronization with the 
processing of the audio stream of Figure 18 . 

Referring now to the flow chart of Figure i?B, processing steps for rate matching 
a live or real tune audio stream origination rate with a processing component or filter 
processing rate is shown. After beginning at step 390, physical time samples are received 
at the processing component or filter at step 392. Again, this can be done by qucryin K a 
clock mechanism or by receiving a timing event notification. With reference to Figure 1 g 
processing of the live audio stream 352 which is ultimately rendered at audio Tenderer filter 
364, the relevant processing component would be the audio Tenderer filler 364 as event 
notifications and control information are received at the input pin instance 36ft. Also, the 
clock mechanism 344 would provide the different timing information. 

From the physical time samples received at step 392, the amount of data processed 
between those time samples can be used to compute or determine a physical time rate at step 
394. The physical time rate is the rate at which the audio hardware actually renders data and 
represents data rendering throughput and is also known as the filter processing rate. If the 
source of the live data stream produces data samples at a faster origination rate than the filter 
processing rate, excess data occurs. If the origination rate is less than the filter processing 
rate, then gaps or starvation for data processing occur. In either case, performance 
degradation results. 

The actual processing rate of the processor that originates the data samples, or 
ordination rate, can be determined from the media or data samples themselves. Because the 
media samples have rime interval information in the form of convention or actual umestamp 
information, media time samples may be computed from the media stream at step 396. This 
allows the origination rate representing the processing rate of the hardware creating the 
media samples to be computed by taking a fixed amount of data processed and dividing this 
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quantity by time it rook for that data to be processed as determined by the media time 
samples. 

Before ending at step 400, the media stream processing is adjusted to match the 
ordination rate to the filter processing rate at step 398. Again, adjustments may occur by 
omitting samples for rendering, adding lime delays between sample rendering, and other 
ways known in the art- 
Referring now to the flow chart shown in Figure 19C, processing steps for using a 
clock mechanism to make a translation is shown. After beginning at step 402, a processing 
component or fiher will receive a correlated time value from a clock mechanism at step 404. 
Note that the processing component or filter will have access to the reference time clock, 
such as a PC clock, that generates the reference time value portion of the correlated time 
value. A media time delta is computed at step 406 by subtracting the media time value from 
the PC clock or other reference time value as found from the correlated rime value received 
previously. The media time delta will be used by the processing component or filter for 
translation and represents the variation of the media time from the PC clock time. 

In order to make the translation, the processing component or filter gets a current 
PC clock time value (or other reference time value) at step 408. The current media time at 
the processing component or filter may be set or timestamp information may be translated 
using the current PC clock time value and the master media time delta at step 410. Since the 
media time delta shows the variation of the media time from PC clock time in the reference 
time basis, it may be simply added to the current PC clock time value in order to arrive at a 
proper "translated" or new time value for timestamp information. After finalizing the 
translation process in step 410, processing ends at step 412. If this communication were 
taking place between two components on a remote processor, such as a DSP, the reference 
time value would be the local host time rather than the PC time. 
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Applying ihe method for translation illustrated in the How chart of Figure MC to the 
interconnected kernel mode filters for processing and rendering a live audio and video stream 
as shown in Figures 1 7 and 18 . it would be the input pin instance 336 on the video 
renderer filter 332 that would receive the correlated time from the clock mechanism 344 at 
step 404. Finally, it would be the video renderer filter 332 that made the computations and 
translation of cither the timestamps on the live video stream 320 or otherwise created a 
translated reference time based on the media time of the clock mechanism 344. 

Those skilled in the art will recognise that in a system of interconnected filters as 
disclosed herein, the uniform buffer allocation facilities and the timing facilities may be used 
together in combination for efficient processing or used separately depending 
implementation problem solution requirements. Furthermore, the timing facilities may be 
used independently of a system of interconnected filters. 

Those skilled in the art will recognize that the methods of the present invention may 
be incorporated as computer instructions stored as a computer program code means on a 
computer readable medium such as a magnetic disk, CD-ROM, and other media common 
in the art or that may yet be developed. Furthermore, important data structures found in 
computer hardware memory may be created due to operation of such computer program code 



means. 



The present invention may be embodied in other specific forms without departing 
from its spirit of essential characteristics. The described embodiments are to be considered 
in all respects only as illustrated and not restrictive. The scope of the invention is, therefore, 
radicated by the appended claims rather than by the foregoing description. All changes 
which come within the meaning and range of equivalency of the claims are to be embraced 
within their scope. 

What is claimed and desired to be secured by United States Letters Patent is: 
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4 - Brief Description of the Drawings 

Figure 1 is a prior art data flow diagram showing a system of interconnected filters 
and drivers under the direction of a controlling agent for bringing sound data from a disk file, 
processing the sound data in some form, and rendering the sound data to be played through 
a speaker. 

Figure 2 shows a system according to the present invention having the same purpose 
as that shown in Figure 1 to read sound data from a disk drive, process that data, and render 
that data to be heard on a speaker, wherein the processing filters and rendering are handled 
by interconnected kernel mode drivers, again under the direction of a controlling agent. 

Figure 3 is a vertical relationship model showing the relationships between driver 
objects, device objects and file objects as created and used in an operating system. 

Figures 4A> 4B and 4C are logical block diagrams of a driver object, device object, 
and file object, respectively, showing their logical relationship with the data structures and 
program code to route messages to appropriate process handling code and to validate the 
creation of new file objects according to the system of the present invention. 

Figure 5 is a flowchart showing the initial set up of the routing and validation 
componentry and the processing of I/O messages by the kernel mode drivers. 
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Figure 6 is a flowchart showing in more detail the processing of a controlling agenL 
the routing and validation mechanisms, and specific create handler routines for creating a 
new file object. 

Figure 7 is a logical diagram showing the horizontal relationship between connected 
filters utilizing the file object structures in an operating system to effectuate such a 
connection in a standardized fashion. 

Figure 8 is a flowchart showing the processing steps taken by a controlling agent in 
user mode lo create and connect the kernel mode filters or drivers of Figure 7 in order to 
effectuate a connection for processing I/O requests received from the controlling agent with 
processing continuing between different drivers (filters). 

Figures 9 and 1 o are logical overview diagrams of the kemel mode drivers and 
connections used to create a chain of kemel mode filters under the direction of a user mode 
controlling agent to implement a system for reading sound data from a hard drive, processing 
the data with the kernel mode filters, and rendering the data to be heard through a speaker. 

Figure 1 1 is a flowchart showing the processing steps for creating the interconnected 
kernel mode drivers for the system shown in Figures 9 and 1 0 . 

Figures 12 and 1 3 illustrate the functioning of a buffer allocator mechanism. 
Figure 1 2 shows a logical arrangement and processing of the allocated buffer frames as 
they are passed from one processing component to another. Figure 1 3 illustrates a buffer 
allocator being represented as a file object that is a "child" of a file object representing an 
input pin instance in a system of interconnected kernel mode filters. Both Figures 1 2 and 
13 illustrate the same filter graph topology. 

Figure 14 shows the buffer allocation in transitions of the system illustrated in 
Figures 9 and 1 0 utilizing buffer allocators for controlling the allocation of buffer frames. 

Figure 1 5 is a flow chart showing the processing steps for bringing data from a disk 
driver through a chain of interconnected kernel mode filters and rendering the data on sound 
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processing hardware specifically showing the operation of buffer allocators and the aciual 
data transferring between buffers for the system shown in Figure I 

Figure 16 is a logical block diagram illustrating how two live data streams can be 
synchronized to a single master clock mechanism. 

Figures 17 and 1 8 axe logical block diagrams showing the live audio system of 
Figure 16 as implemented using the interconnected filter system explained m detail in 
connection with Figure 14. Figure 17 represents the live video stream rendering filters 
which receives a master clock synchronization signal, and Figure 18 illustrates the live 
audio rendering system that has a master clock mechanism for synchronizing both streams 
together and for rate matching the live audio data with the actual audio rendering hardware. 

Figures I9A-19C illustrate how a clock mechanism may be used to synchronize data 
processing of multiple data streams, rate match a stream of daLa with the physical capabilities 
of a hardware renderer, and translate one time base to another using a common time base. 
Figure 1 9 A is a flow chart showing the synchronization processing steps. Figure l?B is a 
flow chart showing the rate matching processing steps. Figure l?C is a flow chart showing 
the translation processing steps. 
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J . Abstract 

A method and computer program product for synchronizing processing between two 
or more data streams (e.g., video and sound input) and for rate matching between two 
different hardware clocks that may drift with respect to one another (e.g. % an originating 
clock represented in a times tamped data stream versus a clock actually rendering the data) 
in a system of interconnected software drivers running id kernel mode. The present 
invention overcomes the coordination complexity and inaccuracies in Ute prior art by 
providing a clocking mechanism in a system wherein multiple drivers having input and 
output connection pin instances arc chained together. The clocking mechanism synchronizes 
between data streams by providing a master clock on an input pin instance of a driver that 
is used to synchronize with other input pin instances on other drivers and "slave" clocks. 
Synchronization is achieved through event notification or stream position queries so that 
corresponding frames of data in separate streams are rendered together (e.g., video frames 
with corresponding sound track). Rate matching is achieved through mouitoring a physical 
clock progression in comparison with a scries of data stream tirnestamps thereby allowing 
adjustments to match the different clock rates- A common physical clock (e.g., PC clock) 
can be used as a reference for a component to translate a particular clock time to a time 
shared by all components with a minimum of error. 

2 . Representative Drawing 
FIG. 2 
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